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FOREWORD 


The  Army  Family  Research  Program  (AFRP)  is  a  5-year  integrated  research 
program  initiated  in  November  1986  in  response  to  research  mandated  by  both 
the  CSA  White  Paper.  1983:  The  Army  Family  and  The  Army  Family  Action  Plans 
(1984-1989).  The  objective  of  the  research  is  to  support  the  Army  Family 
Action  Plans  and  assist  Army  family  programs  and  policies  by  (1)  determining 
the  demographic  characteristics  of  Army  families,  (2)  identifying  motivators 
and  detractors  to  soldiers  remaining  in  the  Army,  (3)  developing  pilot  pro¬ 
grams  to  improve  family  adaptation  to  Army  life,  and  (4)  increasing  opera¬ 
tional  readiness. 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI),  with  assistance  from  the  Research  Triangle  Institute,  Caliber  Associ¬ 
ates,  Human  Resources  Research  Organization  (HumRRO),  and  the  University  of 
North  Carolina,  is  conducting  the  research.  The  research  is  being  sponsored 
by  CFSC  pursuant  to  the  LOA  dated  18  December  1986,  "Sponsorship  ARI  Army 
Family  Research." 

ARI  has  provided  sponsors  frequent  updates  on  the  major  findings  of  this 
research  effort.  This  technical  report  describes  the  development  and  analyses 
of  a  set  of  measures  that  can  be  used  to  assess  the  readiness  of  individual 
soldiers.  The  analyses  resulted  in  the  identification  of  a  comprehensive 
reliable  rating  composite  that  could  be  used  in  further  readiness  research  as 
well  as  operationally  to  measure  individual  readiness.  Future  AFRP  analyses 
will  focus  on  determining  the  relationships  of  family  and  Army  factors  to  this 
measure  of  individual  readiness. 


ANALYSIS  OF  ARMY  FAMILY  RESEARCH  PROGRAM  MEASURES  OF  INDIVIDUAL  READINESS 


EXECUTIVE  SUMMARY 


Requi rement : 

The  Army  Family  Research  Plan  of  the  Chief  of  Staff  of  the  Army  mandated 
research  that  would  explore  how  family  factors  are  related  to  retention, 
readiness,  and  adaptation  to  Army  life.  This  report  supports  that  requirement 
by  exploring  how  soldier  readiness  should  be  conceptualized  and  measured. 


Procedure: 

The  core  of  the  research  was  a  large-scale  field  survey  of  Army  soldiers 
and  their  spouses  and  Army  units  and  installations.  Many  measures  of  individ¬ 
ual  and  family  characteristics,  unit  environment,  Army  policies,  programs,  and 
practices,  soldier  and  spouse  experiences,  needs  and  expectations,  community 
characteristics,  and  perceptions  of  civilian  alternatives  were  collected.  In 
order  to  determine  the  impact  of  these  family  and  Army  factors  on  readiness, 
it  was  necessary  to  identify  one  or  more  reliable  measures  of  individual 
readiness  that  could  be  used  as  the  dependent  variable  in  further  analyses  of 
the  AFRP  survey  or  in  future  operational  or  research  applications  involving 
readiness  measurement. 

A  set  of  candidate  individual  readiness  measures  or  dimensions  was 
initially  identified  through  a  review  of  the  military  readiness  literature.  A 
series  of  workshops  was  held  with  field  grade  officers  and  senior  NCOs  to 
obtain  descriptions  of  critical  incidents  that  occurred  in  their  experience 
and  demonstrated  individual  readiness  or  lack  of  readiness.  These  incidents 
were  analyzed  by  content,  and  a  set  of  12  behaviorally  anchored  readiness 
scales  was  developed  for  use  in  the  AFRP.  In  addition,  a  number  of  other 
readiness  measures  were  developed  by  the  project  team  including  self-ratings 
of  readiness,  numbers  of  awards,  letters,  and  certificates  of  appreciation  and 
commendation,  disciplinary  actions,  and  an  overall  rating  scale  to  be  com¬ 
pleted  by  the  commanders  of  the  soldiers’  units. 
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Another  series  of  workshops  was  conducted  in  order  to  obtain  judgments 
of  experienced  officers  and  NCOs  concerning  the  relative  weight  that  these 
various  measures  should  be  given  in  forming  an  overall  composite  index  of 
individual  readiness.  Comparisons  were  made  of  the  weights  assigned  the 
different  measures  for  forming  readiness  composites  for  combat  and  noncombat 
officers  and  enlisted  personnel. 

Analyses  of  the  readiness  measures  collected  in  the  survey  focused  first 
on  overcoming  missing  data  problems  attributable  mostly  to  the  failure  of  the 
supervisors  of  the  sampled  soldiers  to  rate  their  subordinates  on  the  readi¬ 
ness  scales.  Factor  analyses  of  the  readiness  measures  were  conducted  and  the 
Alpha  reliabilities  of  alternate  readiness  composites  were  obtained.  Finally, 
a  composite  readiness  measure  was  computed  for  each  soldier  in  the  sample  for 
which  adequate  readiness  information  was  available. 


F i  rid i  ngs  : 

•  The  weights  assigned  the  various  readiness  measures  for  forming 
composites  for  officers,  NCOs,  and  junior  enlisted  personnel  in 
combat  and  noncombat  units  were  similar  enough  to  warrant  the  use 
of  a  single  set  of  weights  for  all  soldiers. 

•  The  12  readiness  scales,  which  were  given  the  highest  overall 
weight  assigned  the  various  measures,  constituted  a  single  factor 
on  separate  factor  analyses  of  officer,  NCO,  and  junior  enlisted 
ratings . 

•  The  most  reliable  measure  of  readiness  was  the  average  of  the 
first-  and  second-level  supervisory  ratings  on  the  readiness 
scales. 


Utilization  of  the  Findings: 

The  measures  of  soldier  readiness  that  were  developed  in  this  part  of 
the  research  are  expected  to  be  used  primarily  by  those  in  research  and 
evaluation.  However,  the  readiness  measures  have  also  been  briefed  to 
representatives  from  the  Office  of  the  Deputy  Chief  of  Staff  for  Operations 
(ODCSOPS)  and  the  Office  of  Deputy  Chief  of  Staff  for  Personnel  (ODCSPER)  for 
possible  operational  use. 
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ANALYSIS  OF  ARMY  FAMILY  RESEARCH  PROGRAM 
MEASURES  OF  INDIVIDUAL  READINESS 


Introduction 


Background 

The  Army  Family  Research  Program  (AFRP)  was  initiated  in  Fall  1986  in 
order  to  examine  the  role  that  the  families  of  soldiers  play  in  soldier 
readiness,  retention,  and  sense  of  belongingness  to  the  Army  community.  With 
the  prospect  of  lowered  military  budgets  and  reductions  in  force  on  the 
horizon,  the  Army  was  concerned  with  how  to  attract  and  retain  high  quality 
soldiers  in  an  al 1 -volunteer  force  that  consisted  of  high  propo^+ons  of 
married  military  personnel.  Since  the  family  concerns  of  these  soldiers  could 
have  a  major  impact  on  their  willingness  to  remain  in  the  Army  and  on  their 
level  of  performance,  the  Army  had  instituted  a  number  programs  designed  to 
assist  soldiers  and  their  families  to  adapt  to  Army  life.  However,  hard 
questions  were  being  raised  concerning  the  effectiveness  of  these  family 
programs  and  associated  Army  policies  and  practices  in  maintaining  and  raising 
individual  and  unit  readiness  and  retention.  For  example,  would  the  resources 
spent  on  family  programs  b^  better  spent  on  other  Army  priorities?  How  could 
the  family-related  programs  be  improved?  What  unit  and  installation 
conditions  facilitate  successful  adaptation  to  Army  life?  Do  the  family 
policies,  programs,  and  practices  encourage  high  quality  soldiers  to  remain  in 
the  Army? 

The  AFRP  was  designed  to  provide  information  to  Army  decision  makers 
concerning  these  and  related  questions  and  issues.  The  core  of  the  research 
was  a  large-scale  field  survey  of  Army  soldiers  and  their  spouses  and  Army 
units  and  installations.  The  survey  design  and  development  of  the  instruments 
used  in  the  survey  are  described  in  the  'rRP  report,  Report  on  Survey 
Implementation  (0005AJ)  (1990).  The  plans  for  the  analyses  of  the  survey  data 
ha. 3  been  documented  in  the  report,  AFRP  Analysis  Plan  (1990).  Both  these 
reports  describe  the  instruments  used  to  measure  the  readiness,  retention,  and 
family  adaptation  and  Army  commitment,  the  major  dependent  AFRP  variables  in 
the  planned  analyses.  The  reports  also  describe  the  measures  of  AFRP 
individual  and  family  characteristics,  unit  environment,  Army  policies, 
programs,  and  practices,  soldier  and  spouse  experiences,  needs,  and 
expectations,  community  characteristics,  and  perceptions  of  civilian 
alternatives. 

The  focus  of  this  report  is  on  the  specific  analysis  conducted  for 
the  purpose  of  identifying  one  or  more  reliable  measures  of  individual 
readiness  that  could  be  used  as  the  dependent  variable  in  further  analyses  of 
the  AFRP  survey  data  base.  A  companion  report  will  focus  on  the  analysis 
conducted  to  identify  one  or  more  reliable  measures  of  unit  readiness.  These 
measures  were  developed  by  the  AFRP  primarily  because  of  lack  of  agreement 
within  the  Army  concerning  the  measurement  of  readiness  as  well  as  concern 
over  the  comparability  across  different  types  of  units  and  soldiers  of  the 
existing  measures.  Existing  Army  measures  of  readiness  tend  to  be  unit  (e.g., 
the  Unit  Status  Report)  or  MOS  (e.g.,  the  Skill  Qualification  Test)  specific. 
They  generally  concentrate  on  the  "can  do"  aspects  of  readiness  and  not  the 
"will  do"  or  motivational  component  of  readiness,  whicn  may  be  more  influenced 
by  f ami ly  factors. 
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The  development  of  the  readiness  measures  is  described  in  the  draft 
report,  TR  11:  Definition  and  Measures  of  Individual  and  Unit  Readiness  and 
Family  Phenomena  Affecting  It  (1988).  Further  descriptions  of  the 
developmental  process  are  given  in  the  aforementioned  AFRP  reports. 

Purpose  of  this  Report 

The  primary  purpose  of  this  report  is  to  describe  the  analyses  conducted 
on  the  measures  of  individua1  readiness  that  were  obtained  on  the  soldiers  who 
participated  in  the  Army  Family  Readiness  Project  survey.  The  major  objective 
of  the  analyses  was  to  establish  one  or  more  reliable  measures  of  individual 
readiness  that  could  be  used  either  as  dependent  variables  in  subsequent  AFRP 
analyses  of  the  survey  data  base  or  in  future  operational  or  research 
applications  involving  readiness  measurement.  Decisions  that  were  made  during 
the  analyses  are  explained.  A  concluding  section  presents  the  authors' 
recommendations  concerning  the  use  of  the  available  readiness  measures  in 
current  and  future  research.  Before  describing  the  results,  a  brief  summary 
of  the  development  of  the  measures  is  given. 
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Development  of  the  Individual  Readiness  Measures 


The  procedures  used  to  develop  the  readiness  measures  and  their  time 
frames  are  given  in  Figure  1.  The  developmental  process  was  spread  over  a 
two-year  period  owing  to  difficulties  in  obtaining  required  Army  subject 
matter  expert  input  and  the  necessity  of  coordinating  the  field  tests  of  the 
readiness  measures  with  that  of  other  AFRP  survey  instruments.  The  basic 
approach  in  the  development  of  the  AFRP  instruments  involved  utilizing  the 
judgments  of  experienced  Army  personnel  as  much  as  possible.  This  was 
especially  true  of  the  readiness  measures,  since  their  acceptance  by  the  Army 
was  considered  a  prerequisite  to  the  acceptance  of  findings  concerning  the 
impact  of  family  factors  on  readiness. 

A  literature  review  initiated  the  development  of  the  readiness  measures 
for  the  AFRP.  The  review  focused  on  previous  research  involving  indicators  of 
individual  and  unit  readiness,  spouse  preparedness,  and  family  and  military 
environment  factors  related  to  readiness.  An  initial  list  of  individual  and 
unit  readiness  dimensions  was  constructed  upon  completion  of  the  literature 
review. 

Two  sets  of  workshops  were  then  conducted  to  further  refine  the  initial 
list  of  dimensions.  The  first  series  of  workshops  used  a  critical  incident 
methodology  in  which  officers  and  NCOs  from  both  combat  and  support  units  were 
asked  to  generate  critical  behavioral  incidents  based  upon  their  experience  in 
the  Army.  The  originator  of  each  incident  indicated  the  degree  of  individual 
or  unit  readiness  exemplified  by  the  incident  on  a  nine-point  scale.  The 
incidents  were  analyzed  and  classified  by  AFRP  scientists  who  generated  a  list 
of  readiness  dimensions  for  individuals  and  another  set  of  dimensions  for 
units.  Draft  behavioral ly  anchored  rating  scales  were  constructed  for  each  of 
the  dimensions. 

In  the  second  set  of  workshops,  the  officer  and  NCO  participants  were 
asked  to  try  out  and  evaluate  the  draft  individual  and  unit  readiness  scales. 
The  participants  were  asked  which  dimensions,  when  combined  into  a  composite 
index,  would  produce  the  best  overall  measure  of  the  readiness  of  individual 
officers,  NCOs,  and  junior  enlisted  personnel.  They  were  further  asked  which 
dimensions  were  most  difficult  to  use  in  rating  individuals  and  what  they  felt 
were  the  sources  of  the  difficulty.  The  participants  were  also  asked  to 
identify  types  of  individuals,  if  any,  for  which  they  felt  one  or  more  of  the 
dimensions  would  be  inappropriate  for  use  in  measuring  readiness. 

Upon  completion  of  the  workshops,  a  master  working  list  was  developed  of 
readiness  variables  for  which  measurement  instruments  would  be  developed.  The 
rationale  underlying  the  selection  of  each  variable  involved  three  criteria: 
(1)  it  appeared  reasonable  to  believe  that  high  (or  low)  scores  on  the 
variable  would  be  indicative  of  the  probability  that  an  individual  would 
successfully  complete  his/her  wartime  mission;  (2)  the  variable  would  be 
applicable  to  most  if  not  all  types  of  individuals;  and  (3)  the  variable  could 
be  measured  reliably  and  relatively  easily. 
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Event 


Date 


Literature  Review  to  Identify  Readiness 
Dimensions 

October  1987 

Subject  Matter  Expert  (SME)  Workshops  to 
Generate/Evaluate  Critical  Incidents 

October  -  November  1987 

Development  of  Draft  Scales 

November  -  December 

1987 

First  Pilot  Test  to  Evaluate  Scales 

December  1987 

Revisions  of  Draft  Scales 

January  1988 

Second  Pilot  Test  of  Scales 

February  1988 

Further  Revisions  of  Scales 

March  198 

Field  Test  of  Procedures  and  Instruments 

March  -  April  1988 

Analysis  of  Field  Test  Results 

June  -  August  1988 

Review  and  Approval  of  Readiness  Instruments 
by  CFSC ,  ODCSPER ,  ODSCOPS 

September  -  November 

1988 

Readiness  Weighting  Workshops 

April  -  September  1989 

World-Wide  Administration  of  Readiness 

Measures 

February  -  December 

1989 

Figure  1.  Procedures  Used  to  Develop  the  Readiness  Measures. 


The  preferred  measurement  methodology  for  each  variable  was  then 
determined  by  AFRP  scientists.  Draft  instruments  were  developed  and  field- 
tested.  On  the  basis  of  the  field  test  results,  the  draft  readiness  measures 
were  further  refined.  Finally,  the  instruments  were  reviewed  by  Army 
readiness  proponents  who  declared  them  to  be  adequate  measures  of  individual 
readiness.  The  final  individual  readiness  measures  consisted  of: 

1.  Readiness  rating  scales.  A  set  of  behavioral ly  anchored  rating 
scales  to  be  completed  by  first-  and  second-line  supervisors  of  the  soldiers 
in  the  core  survey  sample. 

2.  An  assessment  of  the  soldier’s  job  performance  relative  to  that  of 
other  soldiers  in  the  unit.  This  assessment  was  to  be  completed  by  the  unit 
commander  for  each  sampled  soldier. 
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3.  A  series  of  self-report  questions  presented  in  the  Soldier 
Questionnaire  that  were  designed  to  collect  information  on  individual 
readiness .  The  questions  query  the  respondent  (the  sampled  soldier)  on 
objectively  verifiable  performance  information  such  as  number  of  awards 
received,  Skill  Qualification  Test  scores,  and  adverse  disciplinary  actions. 

4.  A  series  of  readiness  self-ratino  scales  on  which  the  soldiers 
indicate  their  own  perceived  readiness. 

5.  A  measure  of  the  relative  promotion  rate  of  the  individual  soldiers 
derived  from  Army  records.  Soldiers'  average  time  within  grade  are  compared 
to  that  of  other  soldiers  in  the  same  grade. 

These  measures  are  listed  in  Figure  2.  Appendix  A  presents  copies  of  the 
actual  measures  as  they  appeared  in  the  Soldier  Questionnaire  and  on  the 
rating  forms  used. 

In  anticipation  of  the  requirement  to  form  a  single  comprehensive  measure 
of  individual  readiness  from  the  diverse  measures  obtained,  a  series  of  eight 
workshops  were  held  in  which  the  participants  assigned  weights  to  the 
measures.  Four  of  the  workshops  were  held  in  Europe  and  four  at  Fort  Carson, 
CO.  Judgments  of  71  experienced  officers  and  NCOs  concerning  the  relative 
weight  that  should  be  given  the  available  measures  were  obtained  in  these 
workshops.  The  71  workshop  participants  were  divided  about  equally  between 
combat  and  combat  support  personnel  and  officers  and  NCOs.1  The  participants 
weighted  the  measures  separately  for  E2-4  enlisted  personnel,  NCOs,  and 
officers.  The  specific  instructions  for  assigning  the  weights  are  given  in 
Appendix  B.  The  weights  were  scaled  through  linear  transformations  so  that 
their  sum  totalled  to  100. 

Table  1  presents  the  average  weights  assigned  by  the  workshop 
participants  for  the  measures.  Taken  as  a  set,  the  supervisor  rating  scales 
were  assigned  the  highest  weights  for  all  three  groups  (E2-4,  NCOs,  and 
officers).  There  was,  however,  not  too  much  variation2  in  the  average  weights 
assigned  the  individual  scales,  although  analysis  of  variance  results 
indicated  that  the  mean  weights  assigned  to  the  separate  scales  were 
significantly  different  (p  <  .0001). 

The  second  highest  average  weight  was  assigned  the  unit  commander  ratings 
of  the  relative  performance  level  of  the  soldier  compared  to  other  soldiers  in 
the  unit.  The  high  weight  given  this  measure  and  the  relatively  high  weights 
given  for  the  Enlisted  and  Officer  Evaluation  Reports  indicate  that  the 
workshop  participants  placed  a  high  value  on  supervisory  ratings  in  comparison 
to  the  other  measures. 


1  These  participants  were  chosen  by  the  Army  and  do  not  necessarily 
constitute  representative  samples. 

2  If  a  set  of  scales  are  fairly  highly  intercorrelated,  and  the  weights 
assigned  the  separate  scales  are  not  too  different,  then  a  composite  derived 
through  applying  the  weights  will  be  very  highly  correlated  with  a  composite 
derived  through  simply  summing  or  averaging  the  separate  scale  values  assigned 
the  ratees. 
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1.  Readiness  Ratings  (Completed  by  Supervisors) 

Cooperation/Teamwork/Esprit  de  Corps 
Effort  and  initiative 
General  soldiering  skills 
Individual  deployability  (Army  task/mission) 
Individual  deployability  (personal/fami ly) 

Job  discipline 

Job  technical  knowledge/skills 

Performance  under  pressure  and  adverse  conditions 

*  Care  and  concern  for  subordinates 

*  Care  and  concern  for  subordinates'  families 

*  Leadership  of  subordinates 

*  Maintaining  training  status  of  subordinates 

2.  Unit  commander's  ratings  of  each  sampled  soldier. 

3.  Self-report  performance  items  in  the  Soldier  Questionnaire. 


Question  No. 

38 

44 

**46 

***47/48 

****49 

****50 

51 

52 

54 

55 

56 


Time  taken  off  duty  for  personal  reasons  last  month. 
Last  Physical  Readiness  Test  score. 

Most  recent  Skill  Qualification  Test  score. 

Most  recent  ER/EER  evaluation. 

Most  recent  OER  evaluation. 

Position  in  senior  rater  mass. 

Articles  15  in  past  two  years. 

FLAG  actions  in  past  two  years. 

Letters  of  Appreciation,  etc.,  in  past  two  years. 
Certificates  of  Appreciation,  etc.,  in  past  two  years. 
Awards  and  decorations  received  in  the  military. 


4.  Self-Ratings  of  Readiness  in  the  Soldier  Questionnaire 


Question  No. 

41  Preparedness  for  wartime  job. 

42  Preparedness  for  conflict  using: 

a.  Nuclear  weapons 

b.  Biological  agents 

c.  Chemical  agents 

d.  Conventional  weapons 

43  Comparison  of  job  performance  with  that  of  other  soldiers. 

5.  Promotion  rate  in  comparison  to  other  soldiers  in  same  grade 


*  Obtained  for  officers  and  NCOs  only. 

**  Obtained  from  enlisted  personnel  only. 

***  Obtained  from  NCOs  only. 

****  Obtained  from  officers  only. 


Figure  2.  Individual  Readiness  Measures  Collected  in  Core  Survey. 
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Table  1 


Average  Weights  Assigned  Individual  Readiness  Measures  by  Workshop 
Participants 


Readiness  Measures 

E2-4 

NCO 

Officer 

Supervisory  Ratings 


Cooperation/Teamwork/Esprit  de  Corps 

2.6 

1.8 

1.8 

Effort  and  Initiative 

2.6 

1.9 

2.0 

General  Soldiering  Skills 

3.0 

1.8 

1.8 

Deployability  (Army  Task/Mission) 

2.2 

1.6 

1.6 

Deployability  (Pe^sonal/Fami ly) 

2.1 

1.4 

1.4 

Job  Discipline 

2.6 

1.7 

1.7 

Job  Technical  Knowledge/Skills 

2.8 

2.0 

2.0 

Performance  under  Pressure/Adverse  Conditions 

2.5 

1.7 

1.9 

Care  and  Concern  for  Subordinates 

— 

1.8 

1.8 

Care  and  Concern  for  Subordinates'  Families 

— 

1.6 

1.6 

Leadership  of  Subordinates 

— 

2.0 

2.1 

Maintaining  Training  Status  of  Subordinates 

— 

1.7 

1.6 

Questionnaire  Items 

Time  off  for  personal  reasons  last  month 

12.6 

9.9 

9.7 

Preparedness  for  wartime  job 

Preparedness  for  conflict  using 

3.8 

4.2 

4.1 

a.  Nuclear  weapons 

2.0 

2.2 

2.3 

b.  Biological  agents 

1.8 

2.1 

2.1 

c.  Chemical  agents 

2.6 

2.9 

3.0 

d.  Conventional  weapons 

3.2 

3.5 

3.6 

Comparison  of  job  performance 

2.4 

2.5 

2.1 

Last  SQT  score 

2.6 

2.7 

2.9 

Most  recent  SQT  score 

2.8 

2.9 

— 

Most  recent  ER/EER  evaluation 

— 

2.8 

— 

Most  recent  OER  evaluation 

— 

— 

2.8 

Position  in  senior  rater  mass 

— 

— 

3.0 

Articles  15  in  past  two  years 

1.7 

1.6 

1.5 

FLAG  actions  in  past  two  years 

1.6 

1.5 

1.4 

Letters  of  Appreciation,  etc.,  past  two  years 

1.6 

1.4 

1.4 

Certificates  of  Appreciation,  etc.,  past  two  years 
Number  of  military  awards/decorations 

1.7 

1.5 

1.4 

2.0 

1.9 

2.0 

Other  Measures 

Unit  Commander  Rating 

19.1 

19.1 

20.3 

Promotion  Rate 

17.7 

16.1 

15.0 

SUM 

100.0 

100.0 

100.0 
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Intraclass  correlations  among  the  judges  were  run  across  the  weights. 

(These  correlations  essentially  measure  the  reliability  of  the  mean  weights 
obtained  from  averaging  across  judges  the  weights  assigned  the  separate 
readiness  measures/)  One-rater  and  n-rater  reliabilities  were  computed 
through  use  of  the  Spearman  Brown  formula. 

Table  2  shows  the  results  obtained  for  the  weights  assigned  the  E2-4,  NCO, 
and  officer  readiness  measures,  respectively.  The  very  high  n-rater 
reliabilities  (averaging  .995)  obtained  indicate  that  the  means  of  the  weights 
assigned  the  .arious  readiness  items  by  the  judges  were  quite  stable.  That 
the  sets  of  mean  weights  would  likely  be  highly  similar  across  different 
groups  of  judges  can  be  seen  in  Table  3  which  presents  the  correlations 
between  the  weights  assigned  by  officers  and  by  NCOs.  The  table  also  presents 
the  correlation  between  the  weights  assigned  by  combat,  combat  support,  and 
combat  service  support  judges  and  by  USAREUR  judges  and  Fort  Carson  judges. 
These  correlations  averaged  .986. 


Table  2 

Reliabilities  of  Sets  of  Mean  Weights  Assigned  by  the  Judges 


Type  of  Soldier 

No.  of 
Judges 

No.  of 
Items 

1-rater 

Rel iabi 1 ity 

n-rater 

Reliability 

E2-4 

69 

24 

.783 

.996 

NCO 

70 

29 

.754 

.995 

Officer 

69 

29 

.718 

.994 

Table  3 


Correlations  Between  Sets  of  Mean  Weights  Assigned  by  Different  Groups  of 
Judges  (n  =  number  of  items  or  scales) 


Tvbe  of  Soldier 

E2  -  E4 
(n  =  24) 

NCO 

(n  =  29) 

Officer 
(n=  29) 

NCO  and  Officers 

.988 

.975 

.967 

Combat  and  Support 

.997 

.996 

.988 

USAREUR  and  Fort  Carson 

.982 

.992 

.990 

3  See  Winer,  B.J.  (1971).  Statistical  principles  in  experimental  desion 
(2nd  Ed.).  McGraw-Hill  Publishers;  pg.  290. 
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Of  considerable  practical  importance  is  whether  the  same  sets  of  weights 
could  be  used  in  arriving  at  the  total  readiness  scores  for  the  E2-4,  NCO,  and 
officer  soldiers  in  the  sample.  The  intercorrelations  among  the  weights 
assigned  the  three  types  of  soldiers  were  computed  across  the  23  common  items. 
These  three  correlations  ranged  from  .983  to  .997,  averaging  .990.  An 
intraclass  analysis  was  run  on  the  three  sets  of  mean  weights  of  the  common 
items.  (Here  the  number  of  "judges"  is  3,  and  the  number  of  items  is  23.) 

The  1-rater  reliability  that  resulted  was  .988  and  the  3-rater  reliability  was 
.996. 

These  results  essentially  meant  that  we  could  justify  psychometrically  the 
use  of  the  same  set  of  readiness  weights  for  all  participants  when  forming 
composites  by  obtaining  weighted  averages  across  the  various  readiness  items, 
scales,  and  indexes  available  for  each  type  of  soldier.  This  was  considered  a 
fortuitous  result,  because  applying  a  different  set  of  weights  for  officers, 
NCOs,  and  E2-4s  and/or  combat  and  support  troops  would  have  meant  that  the 
resultant  readiness  scores  would  not  be  rigorously  comparable  across  sampled 
groups. 
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Analysis 


Overview  of  the  Analysis 

The  main  goal  of  the  analysis  was  to  develop  a  composite  measure  of 
individual  readiness  that  would  provide  comparable  scores  for  all  soldiers  in 
the  sample.  That  is,  we  wanted  to  develop  a  set  of  scores  that  allowed 
comparison  of  individual  readiness  across  as  many  of  the  sampled  soldiers  as 
possible.  We  also  wanted  the  composite  readiness  measure  to  be  highly 
reliable  and  to  include  the  separate  measures  of  readiness  that  the  weighting 
workshop  participants  felt  should  receive  the  highest  weights  in  an  overall 
composite. 

Preliminary  analyses  of  the  readiness  measures  based  on  an  initial  subset 
of  the  data  indicated  that  the  most  reliable  composite  that  could  be  obtained 
would  consist  of  the  supervisor  ratings  of  individual  readiness.  The  analyses 
also  indicated  that  because  of  the  relatively  low  correlations  of  the 
supervisor  ratings  with  other  readiness  measures  and  the  relatively  high 
intercorrelations  among  the  rating  scales,  adding  other  measures  to  the 
composite  would  most  likely  lower  the  reliability  of  the  composite. 

Preliminary  analyses  of  the  data  further  indicated  that  sizeable  numbers  of 
cases  in  the  full  sample  would  probably  be  missing  one  or  more  of  the 
measures. 

Based  on  these  early  findings,  the  decision  was  made  to  concentrate  the 
initial  analyses  of  the  full  data  set  on  the  supervisor  readiness  ratings. 
These  ratings,  taken  as  a  group,  had  been  given  the  highest  weight  by  the 
readiness  weighting  workshop  participants  for  forming  an  overall  readiness 
composite.  However,  the  condition  of  the  full  soldier  data  file  when  first 
received  presented  a  number  of  complications  for  the  use  of  the  supervisor 
readiness  ratings  as  the  core  elements  in  a  measure  of  readiness  that  would  be 
comparable  across  all  sampled  soldiers.  To  begin  with,  about  12.5%  of  the 
soldiers  lacked  any  supervisor  ratings.  Of  the  9,659  soldiers  for  whom  at 
least  one  data  element  from  the  individual  readiness  rating  form  ( I RR )  had 
been  recorded  on  the  data  base,  about  6.7%  had  no  usable  designation  of 
whether  the  soldier  had  been  rated  by  a  first-  or  second-line  supervisor. 

About  38.0%  of  the  cases  had  first-line  supervisor  ratings  with  the  status  of 
the  other  rater,  if  any,  unknown.  Similarly,  about  19.2%  of  the  cases  had 
second-line  supervisor  ratings  with  the  status  of  the  other  rater,  if  any, 
unknown.  Only  about  36.2%  of  the  soldiers  had  ratings  that  were  clearly 
designated  as  having  come  from  both  their  first-  and  second-line  supervisors. 
Table  4  presents  the  number  of  sampled  soldiers  having  supervisor  ratings  in 
each  of  these  data  completeness  categories.  As  we  anticipated  that  there 
might  be  systematic  differences  between  the  ratings  given  by  first-and  second- 
line  supervisors,  we  felt  that  some  procedure  aimed  at  trying  to  establish  the 
status  of  all  raters  would  be  required. 

Another  problem  with  the  IRR  data  was  that  some  supervisors  had  handed  in 
rating  forms  but  had  failed  to  rate  the  soldiers  on  one  or  more  of  the 
readiness  scales.  About  1.1%  of  the  9,659  soldiers  for  whom  there  were  any 
data  taken  from  the  readiness  rating  form  actually  had  no  ratings  at  all, 
while  21.5%  of  the  remaining  soldiers  were  missing  one  or  more  ratings  from 
their  data  set. 
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Table  4 

Number  of  Sampled  Soldiers  Having  Supervisor  Ratings 


Number 


Soldiers  on  Soldier  Data  File  11,035 
Soldiers  with  at  least  one  IRR  data  element  9,659 
Soldiers  clearly  having  1st-  and  2nd-line  supervisor  ratings  3,492 
Soldiers  clearly  having  lst-line  supervisor  ratings  1,666 
Soldiers  clearly  having  2nd-line  supervisor  ratings  1,854 
Soldiers  with  unclear  or  unknown  status  of  supervisor(s)  647 


The  missing  data  problems  complicated  the  task  of  developing  a  comparable 
measure  of  individual  readiness  for  all  soldiers.  A  series  of  analyses  were 
conducted  designed  to  compensate  for  the  missing  information.  The  analyses 
initially  concentrated  on  the  impact  on  total  individual  rating  readiness 
scores  of  having  one  or  more  of  the  separate  readiness  scales  missing.  The 
initial  analyses  also  concentrated  on  determining  the  probable  supervisory 
status  of  the  raters  for  whom  it  was  unclear  whether  they  were  first-  or 
second-line  supervisors.  For  this  analysis,  data  from  the  soldiers  for  whom 
there  was  complete  information  from  both  first-  and  second-line  supervisors 
were  used  to  estimate  the  supervisory  status  of  raters  with  incomplete 
information. 

The  next  series  of  analyses  used  data  from  soldiers  with  complete  sets  of 
ratings  from  both  first-  and  second-line  supervisors.  Issues  explored  were: 

1.  Whether  the  factorial  structure  of  the  ratings  was  apparently  the  same 
across  the  first-  and  second-line  supervisors  and  across  soldiers  of 
different  rank  --  E2-E4  enlisted  personnel,  NCOs,  and  officers. 

2.  Should  the  weights  obtained  in  the  readiness  weighting  workshops  be 
applied  to  the  separate  scales  in  deriving  a  composite  overall  score  or 
should  the  simple  arithmetic  sum  or  average  of  the  scales  be  used? 

3.  What  were  the  reliabilities  of  the  first-  and  second-line  supervisor 
readiness  ratings  and  how  did  they  compare  to  the  reliability  of 
ratings  that  were  the  average  of  the  two  supervisor  ratings? 

4.  How  interchangeable  were  first-  and  second-line  supervisory  ratings?  In 
the  absence  of  one  set  of  ratings  should  we  impute  the  other  or  should 
we  impute  the  average  of  the  two  ratings? 

Once  the  above  issues  were  resolved,  and  as  many  soldiers  as  possible  in 
the  sample  had  been  assigned  individual  readiness  scores  based  on  the 
supervisor  ratings,  the  analyses  were  directed  at  the  question  of  the 
relationship  between  the  supervisor  ratings  and  other  measures  of  readiness 
and  performance  in  the  soldier  data  base.  The  factorial  structure  of  the 
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measures  was  explored  and  the  feasibility  of  augmenting  the  supervisor  ratings 
with  other  measures  to  form  a  more  comprehensi ve  overall  measure  of  individual 
readiness  was  determined. 

In  some  of  the  analyses,  sampling  weights  that  had  been  derived  for  the 
soldiers  having  individual  readiness  ratings  were  applied.  (See  Report  on 
Survey  Implementation  (0005AJ)  (1990)  for  a  description  of  the  survey  sampling 
procedure.)  These  weights  were  applied  when  the  interests  of  the  analyses 
went  beyond  the  particular  sample  of  soldiers  obtained  to  the  soldier 
population.  When  the  analyses  were  primarily  focussed  on  the  AFRP  sample  of 
soldiers,  itself,  the  sampling  weights  were  not  applied.  For  example,  the 
weights  were  applied  in  analyses  of  the  factorial  structure  of  the  readiness 
measures;  weights  were  not  applied  in  analyses  aimed  at  imputing4  missing 
readiness  information.  Analyses  in  which  sampling  weights  (the  Soldier  IRR 
weights)  were  used  are  identified  in  the  text. 

With  the  exception  of  comparisons  among  soldiers  by  grade  or  rank  (E2-4 
enlisted  personnel,  NCOs,  warrant  officers  (WO),  and  officers),  no  group 
comparisons  are  presented  in  this  report.  As  individual  readiness  may  be  a 
function  of  many  other  variables,  e.g.,  aptitude  and  education  level,  besides 
membership  in  such  groups  as  married/not  married  or  having  children/having  no 
children,  such  group  comparisons  will  be  made  in  the  future  using  multivariate 
analyses  which  can  adequately  control  for  differences  in  related  variables 
which  may  impact  or  partially  explain  differences  in  readiness  across  various 
groups. 

Using  Composites  in  which  Some  Scale  Values  are  Missing 

As  one  or  more  readiness  scale  values  were  missing  for  approximately  2,050 
soldiers  in  the  sample  (not  including  soldiers  with  no  rating  data  at  all),  an 
analysis  was  run  to  determine  whether  to  use  the  average  of  nonmissing  scale 
values  when  one  or  more  scales  were  missing.  Specif ical ly,  the  analysis 
addressed  the  impact  of  successively  increasing  the  number  of  scales  for  which 
data  were  missing  on  the  correlation  between  an  average  based  on  a  reduced  set 
of  scale  values  and  an  average  based  on  the  complete  set  of  scale  values. 

The  scale  values  from  the  4,491  soldiers  who  had  complete  rating  data  from 
both  supervisors  for  the  first  eight  readiness  scales  were  used  in  this 
simulation  exercise.  The  two  supervisor  ratings  for  each  scale  were  averaged. 
The  average  of  the  eight  readiness  scales  was  obtained  next.  Then  one  scale 
was  dropped  or  assumed  missing  from  each  soldier's  data  set  (each  of  the  eight 
scales  was  dropped  from  1 /8th  of  the  sample  on  a  random  basis)  and  the  average 
of  the  seven  rating  scales  was  obtained.  Then  one  of  the  remaining  seven 
scales  was  dropped  from  each  of  the  eight  groups'  data.  The  scale  dropped 
from  any  one  group's  data  was  selected  randomly  under  the  restriction  that 
once  a  given  scale  had  been  dropped  from  one  group's  data  it  could  not  be 
dropped  from  another  group's  data  that  go  around.  The  average  of  the  six 


4  In  general,  the  authors  avoid  imputing  values  for  variables  that  will 
be  used  as  dependent  variables  in  other  analyses.  However,  in  this  case, 
because  of  the  large  amount  of  missing  data,  we  felt  it  necessary  to  allow  the 
imputation  of  values  when  essentially  we  were  using  the  available  parts  of  a 
composite  dependent  variable  to  impute  the  whole  composite. 
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rating  scales  was  then  obtained.  This  process  continued  until  average  scale 
values  based  on  5,  4,  3,  and  2  scales,  respectively,  were  obtained. 

The  correlations  of  the  averages  based  upon  reduced  numbers  of  scales  with 
the  average  based  on  all  eight  scales  were  then  computed.  As  would  be 
expected,  the  correlations  dropped  as  the  number  of  missing  scale  values 
increased.  But  the  drop  in  the  correlation  was  quite  slow  at  first  and  was 
still  quite  high  (about  .95)  when  the  average  readiness  score  was  based  on 
only  three  scales: 

Number  of  Scales  in  Average 
_7_  _6_  _5_  _4 _ 3  2 

Correlation  with  8-scale  average  .995  .988  .980  .968.948  .905 


On  the  basis  of  these  results,  we  decided  to  obtain  average  readiness  rating 
scale  scores  for  any  cases  that  had  values  for  four  or  more  of  the  eight 
readiness  scales. 

Determining  Rater  Supervisory  Status 

As  mentioned  earlier,  it  was  unclear  in  many  cases  whether  the  rater  of 
the  sampled  soldier  was  a  first-line  or  second-line  supervisor.  Since  planned 
further  analyses  depended  in  part  on  whether  the  rater  should  be  considered  a 
first-line  or  second-line  supervisor,  it  was  decided  to  attempt  to  assign 
first  line/second  line  supervisory  status  to  those  raters  for  whom  such  status 
was  in  doubt.  A  two-stage  process  was  used.  First,  equations  were  developed 
to  estimate  the  supervisor  status  of  the  raters.  Then,  for  those  soldiers 
having  two  raters,  the  estimated  supervisor  status  was  subjected  to  a  series 
of  logic  tests.  The  procedures  employed  are  described  in  more  detail  below. 

Before  attempting  to  assign  supervisory  status  to  any  rater,  a  check  was 
made  to  determine  whether  the  rater  had  assigned  ratings  to  the  ratee  on  fewer 
than  four  of  the  first  eight  scales.  These  rater/ratee  combinations  were 
dropped  from  further  analyses.  Also  dropped  were  rater/ratee  combinations 
where  the  rater  reported  on  the  rating  form  that  he/she  was  not  a  supervisor 
and  was  not  very  familiar  with  the  ratee. 

Using  data  from  the  3,399  remaining  soldiers  having  both  first-  and 
second-line  supervisor  ratings,  multiple  regression  equations  were  computed 
that  estimated  rater  status  from  the  rater's  rank  and  familiarity  with  the 
ratee.  The  equations  were  derived  separately  for  each  enlisted  rank  and  each 
officer  grade  level  and  for  all  warrant  officers  as  a  combined  group  (because 
of  the  small  warrant  officer  sample  size).  These  equations  were  then  used  to 
estimate  whether  raters  for  whom  supervisory  status  was  unclear  were  first-  or 
second-level  supervisors. 

The  overall  accuracy  of  the  regression  equations  in  correctly  predicting 
supervisor  status  was  65%  for  E2-E4  enlisted  personnel,  76%  for  NCOs,  75%  for 
warrant  officers,  and  83%  for  officers.  As  these  percentages  were  not  very 
high,  logic  checks  were  made  for  those  ratees  having  two  sets  of  ratings  and 
for  whom  one  or  both  supervisors'  status  had  been  imputed.  If  one  of  the  two 
raters  was  assigned  the  status  of  first-line  supervisor  and  the  other  the 
status  of  second-line  supervisor,  then  the  status  assignments  were  allowed  to 
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stand.  If,  however,  the  two  raters  had  been  assigned  the  same  status,  then 
the  rater  with  the  lower  rank  was  assigned  the  status  of  first-line  supervisor 
and  the  rater  of  higher  rank  the  status  of  second-line  supervisor.  If  the 
raters  were  of  equal  rank,  then  the  rater  reporting  the  higher  level  of 
familiarity  with  the  ratee  was  assigned  the  status  of  first-line  supervisor. 

If  the  raters'  were  tied  in  both  rank  and  degree  of  familiarity,  then  their 
two  sets  of  ratings  for  the  given  ratee  were  averaged  and  were  considered  to 
have  come  from  one  rater. 

Table  5  shows  the  number  of  soldiers  by  grade  or  rank  for  whom  only  first- 
line  supervisor,  only  second-line  supervisor,  and  both  first-  and  second-line 
supervisor  data  were  available  at  the  conclusion  of  the  rater  designation 
process.  Soldiers  having  ratings  from  any  one  supervisor  on  fewer  than  four 
scales  are  omitted  from  the  table.  It  can  be  seen  that  of  the  original  9,659 
soldiers  listed  on  the  IRR  data  file,  about  98%  had  usable  data  according  to 
the  criteria  adopted. 


Table  5 

Availability  of  Supervisor  Readiness  Rating  Data 


Usable  Supervisor 

Ratinqs 

1st  Line 

2nd  Line 

Both 

Groups 

Only 

Only  1st 

&  2nd 

Total 

E2-4 

1,149 

72  3 

2,393 

4,265 

NCO 

671 

352 

1,594 

2,617 

WO 

47 

22 

115 

184 

Officers 

1,035 

328 

1,036 

2,399 

Total 

2,902 

1,425 

5,138 

9,465 

Factorial  Structure  of 

the  Ratines 

In  order  to  determine  whether  the  readiness  scales  could  be  considered  to 
be  measuring  the  same  basic  underlying  construct,  principal  component  factor 
analyses  were  performed  on  the  eight  scales  that  were  completed  for  all 
soldiers.  The  factor  analyses  were  conducted  separately  for  the  first-line 
supervisors,  the  second-line  supervisors,  and  for  the  average  of  the  first- 
and  second-line  supervisors  ratings.  The  analyses  were  conducted  on  the 
ratings  received  by  the  4,491  soldiers  in  the  sample  who  had  complete  sets  of 
ratings  on  the  eight  scales  from  both  supervisors.  The  soldier  IRR  sampling 
weights  were  applied  in  the  computations. 

In  all  three  factor  analyses,  only  one  factor  met  the  1.0  eigenvalue 
minimum  value  criterion  that  was  used  for  factor  extraction.  Table  6 
presents  the  factor  loadings  obtained  in  the  three  analyses.  The  factor 
loadings  obtained  for  the  scales  were  fairly  similar  across  the  analyses.  In 
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all  three  analysis,  the  Effort  and  Initiative  and  the  Job  Discipline  scales 
had  the  highest  loadings,  while  the  Individual  Deployability  (Personal/Family) 
and  the  Job  Technical  Knowledge/Sk i 11 s  scales  had  the  lowest  loadings. 


Table  6 

Factor  Loadings  Obtained  for  Eight  Readiness  Scales 
(n  =  4,491  soldiers,  all  ranks) 


Scale 

1st  Line 
Supervisor 

Ratinq  Source 
2nd  Line 
Supervisor 

Avg.  1st  &  2nd 
Supervisors 

Cooperation/Team  Work/Esprit 

de  Corps 

.828 

.828 

.864 

Effort  and  Initiative 

.857 

.851 

.883 

General  Soldiering  Skills 

.756 

.751 

.795 

Individual  Deployability 

.705 

.712 

.752 

(Army  Task /Miss ion ) 

Individual  Deployability 

.595 

.570 

.603 

(Personal/Fami ly) 

Job  Discipline 

.868 

.857 

.896 

Job  Technical  Knowledge/Sk i V 

Is 

.523 

.667 

.684 

Performance  under  Pressure  and 

.818 

.814 

.854 

Adverse  Conditions 

The  principal  components  factor  analyses  were  also  run  on  the  12  scales  that 
were  completed  for  NCOs  and  officers.  The  analyses  were  run  separately  on  the 
ratings  obtained  from  the  first-line  supervisors,  the  second-line  supervisors, 
and  the  average  of  the  first-  and  second-line  supervisors  for  the  1,795  soldiers5 
who  had  complete  sets  of  ratings  from  both  supervisors.  The  soldier  IRR  sampling 
weights  were  applied  in  the  computations. 

Again,  in  all  three  analyses,  only  one  factor  met  the  1.0  eigenvalue 
criterion.  Table  7  presents  the  factor  loadings  obtained  in  the  analyses.  The 
factor  loadings  obtained  for  the  scales  were  again  f  n'rly  similar  across  rater 
types.  In  all  three  analyses,  the  four  scales  having  the  highest  loadings  on  the 
single  factor  extracted  were  Leadership  of  Subordinates,  Effort  and  Initiative, 
Job  Discipline,  and  Maintaining  Training  Status  of  Subordinates,  while  the 
Individual  Deployability  (Personal/Family),  Job  Technical  Knowledge/Skills,  and 
Individual  Deployability  (Army  Task/Mission)  scales  had  the  lowest  loadings. 

The  similarity  of  the  factor  analytic  results  for  the  first-  and  second-line 
supervisors  suggests  that  the  scales  are  measuring  the  same  construct  for  both 
types  of  raters.  The  similarity  of  the  results  obtained  with  the  eight  scales 


5  These  1,795  soldiers  are  a  subset  of  the  4,491  soldiers  who  had 
complete  data  for  the  first  eight  scales. 
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and  with  the  twelve  scales,  suggests  that  a  measure  of  individual  readiness  based 
on  the  12  scales  would  essentially  be  measuring  the  same  underlying  construct  as 
a  measure  based  on  the  eight  scales. 

It  is  interesting  to  note  that  the  factor  loadings  obtained  for  the  average 
of  the  first-  and  second- line  supervisors  were  consistently  higher  (with  but  one 
exception)  across  all  scales  in  both  sets  of  analyses  than  the  factor  loadings 
obtained  for  the  separate  ratings  made  by  the  first-  and  second-line  supervisors. 
Though  the  differences  between  the  loadings  are  marginal,  the  higher  loadings 
suggest  that  the  averages  of  the  two  supervisor  ratings  can  provide  better 
measures  of  the  underlying  construct  than  either  supervisor's  ratings  alone.  (As 
seen  on  page  19,  the  Alpha  reliabilities  of  composites  derived  from  averaging  the 
first-  and  second-line  supervisor  ratings  were  higher  than  the  reliabilities  of 
composites  derived  from  either  the  first-  or  second-line  supervisor  ratings 
alone. ) 


Table  7 

Factor  Loadings  Obtained  for  Twelve  Readiness  Scales 
(n  =  1,795  NCOs  and  Officers) 


Rating  Source 

Scale 

1st  Line 
Supervisor 

2nd  Line 
Supervisor 

Avg.  1st  &  2nd 
Supervisors 

Cooppration/Team  Work/Esprit  de  Corps 

.828 

.805 

.855 

Effort  and  Initiative 

.858 

.846 

.886 

General  Soldiering  Skills 

.757 

.700 

.761 

Individual  Deployability 

.650 

.698 

.710 

(Army  Task/Mission) 

Individual  Deployability 

.565 

.555 

.575 

(Personal/Fami ly) 

Job  Discipl  ine 

.850 

.849 

.880 

Job  Technical  Knowledge/Skills 

.470 

.683 

.677 

Performance  under  Pressui a  and 

.816 

.804 

.848 

Adverse  Conditions 

Care  and  Concern  for  Subordinates 

.818 

.821 

.858 

Care  and  Concern  for  Subordinates' 

.748 

.778 

.792 

Fami 1 ies 

Leadership  of  Subordinates 

.881 

.874 

.914 

Maintaining  Training  Status  of 

Subordinates 

.852 

.844 

.880 

As  the  Individual  Deployability  (Personal/Family)  scale  in  general  had  the 
lowest  factor  loading  on  the  one  common  factor  obtained  in  the  six  factor 
analyses,  consideration  was  given  to  dropping  this  scale  from  composite 
measures  consisting  of  the  sum  or  average  of  the  scales.  The  low  loadings 
suggested  that  the  scale  would  marginally  (if  not  negatively)  contribute  to 
the  reliability  of  composites  based  on  the  other  scales.  But  more 
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importantly,  the  scale's  presence  in  an  individual  readiness  composite  could 
be  interpreted  as  biasing  the  measure  toward  one  that  would  be  more  related  to 
family  factors.  Though  this  scale  had  its  origins  in  the  critical  readiness 
incidents  provided  by  experienced  Army  NCOs  and  officers  and  would,  with 
little  doubt,  measure  an  aspect  of  individual  readiness,  it  was  decided  to 
drop  the  scale  from  the  readiness  composite  in  the  interest  of  avoiding  future 
controversy. 

Choosing  between  Weighted  and  Unweighted  Composites  Readiness  Scores 

Analyses  were  performed  to  help  determine  whether  the  weights  obtained  in 
the  readiness  weighting  workshops  should  be  applied  to  the  separate  readiness 
scales  in  arriving  at  the  composite  readiness  scores  or  whether  simple 
arithmetic  averages  of  the  scales  should  be  used.  Two  types  of  statistics 
were  obtained.  First,  the  correlations  between  the  average  readiness  scores 
were  obtained  using  both  weighted  and  unweighted  scales.  These  correlations 
were  derived  separately  for  the  E2-4  personnel,  NCOs,  warrant  officers,  and 
officers  for  the  weighted  and  unweighted  composites  obtained  from  the  first- 
line  supervisors,  the  second-line  supervisors,  and  the  average  of  the  first- 
and  second-line  supervisors  (only  cases  having  complete  sets  of  ratings  were 
used).  These  correlations  were  extremely  high,  ranging  from  .9993  to  .9998. 
(Very  high  correlations  between  weighted  and  unweighted  scores  are  frequently 
obtained  when  the  component  items  or  scales  are  fairly  highly  intercorrelated 
and  the  weights  are  not  extremely  different--see  Footnote  2,  page  5.) 

A  second  analysis  compared  the  Alpha  reliabilities6  of  composites 
consisting  of  the  weighted  sum  of  the  separate  scale  scores  and  composites 
formed  through  the  simple  summation  of  the  unweighted  scale  scores.  The  Alpha 
coefficients  were  derived  separately  for  the  composites  obtained  for  E2-4 
personnel,  NCOs,  warrant  officers,  and  officers  for  the  first-line 
supervisors,  the  second-line  supervisors,  and  the  average  of  the  first-  and 
second-line  supervisors.  The  Soldier-IRR  sampling  weights  were  applied  in 
calculating  the  Alpha  coefficients. 

Table  8  shows  the  obtained  Alpha  reliabilities  broken  down  by  group  and 
rater  type  for  the  weighted  and  unweighted  composites.  Inspection  of  the 
table  shows  that  the  unweighted  composites  were  slightly  but  consistently  more 
reliable  than  the  comparable  weighted  composites.  As  the  correlations  between 
the  weighted  and  unweighted  composites  were  extremely  high  (above  .999)  and 
the  reliabilities  slightly  favored  the  unweighted  composites,  it  was  decided 
to  use  the  unweighted  composites  (or  their  equivalent  average  scale  values)  in 
further  analyses  of  the  individual  readiness  rating  data.  The  unweighted 
composites  have  the  further  advantage  of  being  easier  to  obtain  in  a  field 
setting.  Moreover,  not  using  the  weights  avoids  future  questions  concerning 
the  derivation  and  application  of  the  weights  themselves. 


6  An  Alpha  reliability  is  an  internal-consistency  measure  of  reliability 
which  is  essentially  based  on  the  ratio  of  the  sum  of  the  variance  of 
component  scores  to  the  variance  of  a  total  score  based  on  the  sum  of  the 
component  scores. 
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Table  8 


Alpha  Reliabilities  of  Unweighted  and  Weighted  Composites 


Group 

First-Line  Rater 

Second-Line  Rater 

Averaae  of  Both  Raters 

UnWted 

Wted 

UnWted 

Wted 

UnWted 

Wted 

E2-4 

.898 

.896 

.908 

.907 

.923 

.920 

NCO 

.948 

.945 

.950 

.948 

.960 

.958 

WO 

.918 

.915 

.932 

.930 

.936 

.933 

Officers 

.919 

.915 

.930 

.926 

.937 

.933 

Choosinq  Between  First- 

and  Second-Line  Raters  or  Their  Averaae 

As  noted  earlier,  a  large  number  of  the  sampled  soldiers  were  only  rated 
by  one  supervisor.  In  fact,  less  than  55%  of  the  9,465  soldiers  with  usable 
rating  data  were  rated  by  both  the  first-  and  second-line  raters.  In  view  of 
the  extent  of  the  missing  data,  an  issue  that  obviously  needed  to  be  addressed 
was  whether  to  use  the  first-  and  second-line  raters'  data  interchangeably 
when  the  data  from  one  rater  were  missing.  Supporting  the  use  of  simple 
substitution  of  one  rater's  values  for  the  other  were  the  results  of  the 
factor  analyses  described  earlier.  The  factor  loadings  of  the  rating  scales 
were  highly  similar  across  both  raters  for  E2-4  personnel,  NCOs,  and  officers 
(see  Tables  6  and  7).  The  Alpha  reliabilities  of  their  composites  were 
similarly  high,  averaging  about  .92  for  the  first-line  supervisor  ratings  and 
.93  for  the  second-line  supervisor  ratings. 

However,  the  mean  ratings  given  the  first-  and  second-line  supervisors 
differed  somewhat.  Table  9  presents  the  mean  ratings  given  the  E2-4  enlisted 
personnel,  NCOs,  warrant  officers,  and  officers.  These  means  were  calculated 
applying  the  Soldier  IRR  weights.  A  repeated  measure  analysis  of  variance 
using  the  general  linear  model  procedure  with  sampling  weights  was  run  to  test 
the  significance  of  the  mean  differences.  The  rater  type  means  were  not 
significantly  different,  but  both  the  group  mean  differences  and  the 
interaction  group  x  rater  type  term  were  significant  at  the  .0001  level.  That 
the  group  means  were  highly  significantly  different  is  not  unexpected--one 
would  expect  higher  ranked  personnel  to  be  given  higher  readiness  ratings  on 
the  average  than  lower  ranked  personnel.  The  significant  interaction  term 
indicates  that  the  amount  of  the  difference  in  mean  ratings  between  the  first- 
and  second-line  supervisors  varied  across  the  four  groups.  For  the  E2-4  and 
officers,  the  second-line  supervisor  average  ratings  were  higher  than  the 
first-line  supervisor  average  ratings;  for  the  NCOs  and  warrant  officers,  the 
first-line  supervisor  average  ratings  were  higher  than  those  of  second-line 
supervisors.  This  finding  suggested  that  simply  using  the  two  raters'  data 
interchangeably  could  introduce  systematic  bias  in  the  resultant  individual 
readiness  scores  depending  upon  the  group  the  soldier  was  in  and  whether 
first-line  supervisor  ratings  were  being  substituted  for  second-line 
supervisor  ratings  or  vice  versa. 
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Table  9 

Means  of  First-  and  Second-Line  Supervisor  Ratings 


Group 

1st  Line 

2nd  Line 

Average 

E2-4 

5.118 

5.183 

5.150 

NCO 

5.802 

5.702 

5.752 

Warrant  Officers 

6.082 

5.932 

6.007 

Officer 

6.153 

6.195 

6.174 

Total  Sample 

5.501 

5.514 

5.508 

Moreover,  the  correlations  between  the  average  of  the  seven  or  eleven 
ratings  given  by  the  first-line  supervisors  and  the  average  of  the  comparable 
ratings  given  by  the  second-line  supervisors  were  disappointingly  low.  Table 
10  gives  these  correlations  for  E2-4  enlisted  personnel,  NCOs,  warrant 
officers,  and  officers  separately.  (The  correlations  were  computed  using  only 
the  cases  for  whom  there  were  complete  data.  The  Soldier-IRR  sampling  weights 
were  applied.)  The  correlation  between  the  two  average  ratings  ranged  from 
.35  for  warrant  officers  to  .52  for  E2-4  and  were  far  too  low  to  justify  using 
the  first-  and  second-line  supervisor  ratings  essentially  as  parallel  scores, 
substituting  one  for  the  other. 


Table  10 

Correlations  Between  First-  and  Second-Line  Supervisor  Ratings 


Group 

n 

1st  Line 
with 

2nd  Line 

1st  Line 
with  Average 
Both  Raters 

2nd  Line 
with  Average 
Both  Raters 

E2-4 

2,118 

.52 

.87 

.87 

NCO 

1,041 

.48 

.86 

.86 

Warrant  Officers 

72 

.35 

.82 

.82 

Officers 

700 

.45 

.86 

.84 

The  decision  was  made  therefore  to  use  the  average  of  the  first-  and 
second-line  supervisors'  ratings  as  the  primary  basis  of  the  individual 
readiness  scores.  The  factor  loadings  of  the  rating  scales  when  rater 
averages  were  used  were  generally  higher  than  comparable  loadings  for  the 
first-line  and  second-line  ratings  factor  analyzed  separately  (see  Tables  5 
and  6).  The  Alpha  reliabilities  of  the  composites  formed  by  averaging  the  two 
supervisor  ratings  were  consistently  higher  than  the  reliabilities  obtained 
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for  either  the  first-or  second-line  raters  (see  Table  9).  These  Alpha 
reliabilities  were  .923  for  the  E2-4  personnel  (7  scales)  and  averaged  .944 
for  the  NCOs,  warrant  officers,  and  officers  (11  scales). 

The  correlations  between  the  average  of  the  first-line  supervisor  ratings 
and  the  average  of  the  first-  and  second-line  supervisor  ratings  were  quite 
high,  averaging  about  .85  across  the  four  groups.  The  correlations  between 
the  average  of  the  second-line  supervisor  ratings  with  the  average  cf  first- 
and  second-line  supervisor  were  equally  high  (see  Table  10).  These  high 
correlations,  as  well  as  the  factorial  and  reliability  results  mentioned 
above,  led  to  the  selection  of  the  average  of  the  two  supervisor's  ratings  as 
the  individual  readiness  measure  of  choice. 

Imputation  of  Soldier  Readiness  Scores 

Multiple  regression  equations  were  used  to  estimate  or  impute  the  average 
of  the  first-  and  second-line  supervisors'  readiness  ratings  for  those  sampled 
soldiers  for  whom  there  was  only  one  rater.  The  equations  were  derived 
separately  for  the  sampled  E2-4  personnel,  NCOs,  warrant  officers,  and 
officers.  The  regression  equations  were  derived  using  data  from  the  soldiers 
for  whom  there  were  two  complete  sets  of  ratings  on  the  scales.  Separate 
equations  were  derived  for  estimating  the  average  of  the  first-  and  second- 
line  supervisors'  readiness  ratings  when  the  first-line  supervisor  ratings 
were  missing  and  when  the  second-line  supervisor  ratings  were  missing.  The 
independent  variables  used  in  the  equations  to  estimate  the  average  ratings 
when  the  second-line  supervisor  ratings  were  missing  were  first-line 
supervisor  ratings  (average  of  relevant  scales),  first-line  supervisor's  rank, 
and  ratee  rank.  When  the  first-line  supervisor  ratings  were  missing,  the 
independent  variables  were  second-line  supervisor  ratings  (average  of  relevant 
scales),  second  line  supervisor's  rank,  and  ratee  rank.  The  dependent 
variable  in  both  cases  was  the  average  of  the  first-  and  second-line 
supervisors  ratings  taken  across  the  relevant  scales. 

A  multiple  regression  approach  for  accomplishing  the  imputation  of  the 
average  rating  was  chosen  in  lieu  of  a  "hot  deck"  procedure  because  it  was 
anticipated  that  the  part/whole  relationship  between  the  principal  independent 
variable  (the  supervisor  ratings  that  were  available)  and  the  average  of  the 
supervisor's  ratings  would  produce  sizeable  multiple  correlation  coefficients 
(R).  In  fact,  the  multiple  correlation  obtained  for  the  four  groups  (E2-4, 
NCOs,  warrant  officers,  and  officers)  ranged  from  .814  to  .875  (see  Table  11). 

After  the  equations  were  used  to  estimate  or  impute  first-  and  second-line 
Average  Supervisor  Ratings  for  the  sampled  soldiers  that  only  had  ratings  from 
one  supervisor,  a  dummy  variable  was  created  which  indicated  whether  the 
soldier  had  an  imputed  value  or  an  actual  value  for  his/her  individual 
readiness  score.  This  variable  can  be  used  to  control  for  imputation  process 
effects  if  subsequent  research  indicates  that  is  necessary. 
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Table  11 


Statistics  for  Equations  Used  to  Impute  Average  of  First-  and  Second-Line 
Supervisor  Readiness  Scores 


Group 

No. 

First-Line  Supervisor 

Data  Available 

Second-Line  Supervisor 
Data  Available 

R2 

R 

Mean  Sq 
Error 

R2 

R 

Mean  Sq. 
Error 

E2-4 

2118 

.7649 

.875 

.2420 

.7645 

.874 

.2425 

NCO 

1041 

.7512 

.867 

.2028 

.7476 

.864 

.2065 

WO 

72 

.6623 

.814 

.1534 

.6947 

.833 

.1387 

Officer 

700 

.7624 

.873 

.1093 

.7145 

.845 

.1313 

Relationships  Between  Supervisor  Ratinas  and  Other  Performance/Readiness 

Measures 


The  correlations  between  Average  Supervisor  Rating  and  other  performance/ 
readiness  measures  collected  in  the  AFRP  survey  are  shown  in  Table  12  for  each 
of  the  four  groups  and  for  the  total  sample.  The  sample  Soldier-IRR  weights 
were  used  in  calculating  these  correlations.  The  number  of  cases  whose  data 
were  used  in  the  computations  varies  considerably  both  across  the  measures  and 
between  the  groups  because  of  missing  data  and  because  some  of  the  measures 
were  not  applicable  to  two  or  more  groups.  The  measures  with  the  most  missing 
data  (5%  or  more)  are  listed  in  Table  13. 

It  can  be  seen  through  examination  of  both  Table  12  and  Table  13  that  the 
readiness/performance  measures  that  are  related  more  highly  to  Average 
Supervisor  Rating  generally  have  higher  amounts  of  missing  data.  This  is 
especially  true  of  the  measures  that  are  based  on  supervisor  ratings.  This 
result  is  unfortunate  since  these  other  ratings  would  be  likely  candidates  to 
add  to  the  Average  Supervisor  Rating  in  order  to  increase  the 
comprehensiveness  of  the  AFRP  basic  measure  of  individual  readiness  without 
unduly  lowering  the  reliability  of  such  a  composite  measure.7  But  it  would  be 
difficult  to  maintain  comparability  of  total  readiness  scores  across  all  the 
sampled  soldiers  if  the  added  measures  were  missing  or  not  applicable  to  large 
percentages  of  the  soldiers. 

The  correlations  of  Average  Supervisor  Rating  with  the  nonsupervisory 
measures  were  generally  fairly  low,  but  in  the  expected  direction.  For 
example,  Articles  15  and  FLAG  actions  had  negative  correlations  with  Average 
Supervisor  Rating  that  ranged  from  -.04  to  -.21  across  the  four  groups.  Last 
Physical  Readiness  Test  score,  Most  recent  SQT  score,  Letters  of  appreciation, 


7  In  general,  the  higher  the  intercorrelations  among  component  variables, 
the  higher  the  Alpha  reliability,  other  factors,  e.g.,  variable  weights  and 
variance,  being  equal. 
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Table  12 


Correlations  of  Average  Supervisor  Rating  with  Performance  Variables 
by  Grade  Level 


Variable* 

E2-4 

NCO 

HO 

Officers 

Total 

Group 

S38T** 

No.  of  types  of  activities  took  time  off  for 

-.08 

-.06 

-.01 

.05 

-.04 

S41 

Prepared  to  perform  wartime  job  tasks 

.11 

.10 

.15 

.13 

.16 

S42A 

Task  preparedness  —  nuclear  weapons 

.02 

.00 

.09 

.07 

-.05 

S42B 

Task  preparedness  --  biological  agents 

.00 

-.02 

.00 

.02 

-.03 

S42C 

Task  preparedness  —  chemical  agents 

.01 

-.01 

.06 

.04 

-.01 

S42D 

Task  preparedness  —  conventional  weapons 

.05 

.06 

.07 

.12 

.07 

S43 

Pay  grade  job  performance  comparison 

.19 

.19 

.17 

.17 

.23 

S44 

Last  Physical  Readiness  Test  Score 

.09 

.10 

-.01 

.12 

.13 

S46 

Most  recent  SQT  score 

.19 

.18 

N/A 

N/A 

.24 

S47 

Most  recent  ER/EER  evaluation 

N/A 

.36 

N/A 

.  N/A 

.36 

S49 

Most  recent  OER  evalution 

N/A 

N/A 

.40 

.39 

.38 

S50 

Last  rating  above/below  center  mass 

N/A 

N/A 

.15 

.16 

.15 

S51 

Articles  15  received  past  two  years 

-.15 

-.10 

-.04 

-.21 

-.21 

S52 

FLAG  Actions  received  past  two  years 

-.06 

-.12 

-.08 

-.17 

-.15 

S54 

Letters  of  Appreciation,  etc.,  past  two  years 

.05 

.06 

.06 

.08 

.07 

S55 

Certificates  of  Appreciation  past  two  years 

.07 

.02 

-.09 

.05 

.01 

S56 

Awards  and  decorations  received  while  in  military 

.09 

.16 

.04 

.19 

.26 

CMD  RATE  Commander  assessment  of  job  performance 

.35 

.43 

.51 

.46 

.45 

*  The  variable,  Promotion  rate,  was  unavailable  at  the  time  these  analyses  were  run. 

**  Variable  was  transformed  to  number  of  different  activities  for  which  the  soldier 
took  time  off  duty  last  month. 

Table 

Percen 

13 

t  of  Cases  Having  Missing  Data  on 

Selected  Performance/Readiness  Measures 

Variable 

E2-4 

(n-4265) 

NCO 

(n-2617) 

HO 

(n-184) 

Officers 

(n-2399) 

Total 

(n-9465) 

S44 

Last  Physical  Readiness  Test  Score 

26 

13 

17 

15 

19 

S46 

Most  recent  SQTest  score 

32 

13 

-- 

— 

25* 

S47 

Most  recent  ER/EER  evaluation 

-- 

40 

-- 

-- 

40* 

S49 

Most  recent  OER  evaluation 

-- 

— 

11 

6 

7* 

S50 

Last  rating  above/below  center  mass. 

— 

20 

22 

22* 

CMD  RATE 

Commander  assessment  of  job  performance 

32 

32 

27 

55 

38 

*  Percent  based  on  total  number  of  cases  for  whom  the  measure  Is  applicable. 
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and  Awards/decorations  had  small  positive  correlations  with  Average  Supervisor 
Rating.  It  is  interesting  to  note  that  the  self-ratings  of  preparedness  to 
carry  out  assigned  tasks  in  a  war  in  which  the  enemy  used  nonconventional 
weapons  (S42A,  B,  C)  generally  had  no  relationship  with  Average  Supervisor 
Rating.  However,  self-ratings  of  preparedness  to  perform  wartime  tasks  in  a 
presumed  more  conventional  warfare  setting  (S41  and  S42D)  generally  had  low 
positive  correlations  with  Average  Supervisor  Rating. 

In  order  to  determine  whether  some  of  the  performance/readiness  measures 
could  be  considered  measures  of  the  same  underlying  construct,  separate 
principal  components  factor  analyses  of  the  intercorrelation  matrices  of  the 
measures  were  conducted  for  each  of  three  grade  groups  (E2-4,  NCO,  and 
officers).  In  the  analyses,  factors  with  eigenvalues  greater  or  equal  to  1.0 
were  rotated  using  the  varimax  routine.  Soldier-IRR  weights  were  applied  in 
the  calculation  of  the  correlation  matrices.  The  rotated  factor  patterns  are 
shown  in  Tables  14A,  B,  and  C. 

Examination  of  the  factor  loadings  obtained  in  the  analyses  indicated  that 
four  groups  of  variables  loaded  heavily  on  four  factors  in  each  analysis.  In 
each  of  the  three  analyses,  Soldier's  Questionnaire  items  42A,  B,  and  C,  which 
are  measures  of  the  soldier's  self-ratings  of  their  preparedness  to  fight  in  a 
war  in  which  the  enemy  uses  nonconventional  weapons,  loaded  fairly  heavily  (at 
or  above  .40)  on  one  factor.  Similarly,  in  each  analysis,  items  41  and  42D, 
which  are  apparently  measures  of  soldier  self-ratings  of  preparedness  to  fight 
in  a  conventional  war,  loaded  fairly  heavily  on  a  factor.  Also  in  all  three 
analyses,  items  S54  and  S55  had  fairly  high  loadings  on  a  factor  that 
apparently  is  measuring  tendency  to  receive  letters  and  certificates  of 
appreciation  and  commendation. 


Table  14A 

Rotated  Factor  Pattern  for  Performance/Readiness  Measures 
(n  =  1,403  E2-4  soldiers) 


Variable 

I 

II 

III 

Factor 

I 

V 

VI 

S38T 

No.  of  types  of  activities  took  time  off  for 

03* 

01 

-16 

03 

-11 

12 

S41 

Preparedness  for  wartime  job 

26 

06 

02 

02 

70 

14 

S42A 

Task  preparedness  —  nuclear  weapons 

69 

-02 

04 

07 

07 

12 

S42B 

Task  preparedness  --  biological  agents 

95 

02 

01 

00 

08 

01 

S42C 

Task  preparedness  --  chemical  agents 

91 

-01 

-02 

-04 

14 

03 

S42D 

Task  preparedness  --  conventional  weapons 

43 

06 

01 

-02 

43 

04 

S43 

Pay  grade  job  performance  comparison 

-01 

08 

21 

02 

27 

26 

S44 

Last  Physical  Readiness  Test  Score 

04 

08 

00 

-08 

01 

32 

S46 

Most  recent  SQT  score 

01 

02 

16 

00 

15 

34 

S51 

No.  of  Articles  15  received  past  two  years 

-01 

05 

-28 

40 

02 

05 

S52 

No.  of  FLAG  Actions  received  past  two  years 

04 

-02 

-04 

80 

-01 

-11 

S54 

No.  of  letters  appreciation  past  two  years 

-02 

65 

-01 

-01 

04 

10 

S55 

No.  of  certificates  appreciation  past  two  years 

01 

75 

03 

01 

05 

12 

S56 

No.  of  awards/decorations  while  in  military 

07 

25 

07 

08 

03 

30 

CMD  RATE 

Commander  assessment  of  job  performance 

01 

04 

51 

-09 

02 

08 

AVG  IRR 

Average  Supervisor  Ratings 

04 

00 

64 

-06 

02 

19 

*  Decimal  points  omitted. 
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Table  14B 


Rotated  Factor  Pattern  for  Performance/Readiness  Measures 
(n  =  763  NCOs) 


Factor 

Variable 

“1 - 

TT - 

TTT 

Tv 

V 

VI 

S38T 

No.  of  types  of  activities  took  time  off  for 

-02* 

00 

03 

-04 

06 

69 

S41 

Prepared  to  perform  wartime  job  tasks 

27 

07 

07 

48 

-07 

-09 

S42A 

Task  preparedness  —  nuclear  weapons 

74 

05 

03 

13 

-03 

-02 

S42B 

Task  preparedness  —  biological  agents 

92 

-03 

08 

14 

-01 

-01 

S42C 

Task  preparedness  —  chemical  agents 

88 

-03 

05 

24 

04 

-01 

S42D 

Task  preparedness  —  conventional  weapons 

37 

03 

01 

66 

02 

-08 

S43 

Pay  grade  job  performance  comparison 

08 

32 

10 

22 

-22 

-08 

S44 

Last  Physical  Readiness  Test  Score 

-03 

15 

20 

-13 

-19 

-05 

S46 

Most  recent  SQT  score 

-04 

33 

-03 

36 

-01 

08 

S47 

Most  recent  ER/EER  evaluation 

03 

52 

03 

18 

-34 

04 

S51 

No.  of  Articles  15  received  past  two  years 

02 

-17 

05 

-08 

33 

-02 

S52 

No.  of  FLAG  Actions  received  past  two  years 

-03 

-06 

-05 

01 

60 

07 

S54 

No.  of  letters  appreciation  past  two  years 

08 

07 

82 

12 

03 

04 

S55 

No.  of  certificates  appreciation  past  two  years 

06 

01 

75 

09 

-03 

01 

S56 

No.  of  awards/decorations  while  in  military 

07 

12 

10 

24 

-04 

02 

CMD  RATE 

Coimander  assessment  of  job  performance 

01 

63 

06 

-02 

-17 

-03 

AVG  1RR 

Average  Supervisor  Ratings 

-02 

67 

03 

12 

-02 

00 

*  Decimal 

points  omitted. 

Table  14C 

Rotated 

Factor  Pattern  for  Performance/Readiness  Measures 

(n  =  707  officers) 

Factor 

Variable** 

I 

11 

III 

IV 

V 

VI 

S38T 

No.  of  types  of  activities  took  time  off  for 

02* 

-04 

06 

00 

-03 

64 

S41 

Prepared  to  perform  wartime  job  tasks 

18 

07 

06 

10 

58 

-05 

S42A 

Task  preparedness  --  nuclear  weapons 

80 

00 

03 

03 

19 

06 

S42B 

Task  preparedness  —  biological  agents 

90 

-03 

06 

01 

08 

-02 

S42C 

Task  preparedness  --  chemical  agents 

92 

00 

00 

00 

18 

-02 

S42D 

Task  preparedness  —  conventional  weapons 

34 

03 

-01 

03 

70 

09 

S43 

Pay  grade  job  performance  comparison 

01 

30 

06 

09 

22 

-11 

S44 

Last  Physical  Readiness  Test  Score 

-01 

00 

10 

20 

20 

-10 

S49 

Most  recent  OER  evaluation 

-03 

65 

05 

35 

05 

07 

S50 

Last  rating  above/below  center  mass 

00 

82 

03 

11 

-04 

-02 

S54 

No.  of  letters  appreciation  past  two  years 

01 

10 

83 

-01 

-03 

04 

S55 

No.  of  certificates  appreciation  past  two  years 

09 

-01 

57 

07 

08 

00 

S56 

No.  of  awards/decorations  while  in  military 

-08 

05 

19 

13 

13 

07 

CMD  RATE 

Conrander  assessment  of  job  performance 

00 

22 

02 

55 

08 

-03 

AVG  IRR 

Average  Supervisor  Ratings 

05 

18 

08 

72 

07 

05 

*  Decimal  points  omitted. 

**  The  disciplinary  measures,  S51  and  S52  were  omitted  from  this  analysis  owing  to  their  lack  of  variance 
in  the  subset  of  officers  having  complete  data  on  the  remaining  variables. 
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The  measure  of  primary  interest  in  this  analysis,  Average  Supervisor 
Rating,  also  helped  define  a  factor  in  each  of  the  analysis.  In  the  analysis 
of  the  E2-4  data,  the  average  rating  measure  and  the  Commander's  assessment  of 
job  performance  (CMD  RATE)  loaded  heavily  on  one  factor.  In  the  NCO  sample, 
one  of  the  factors  had  high  loadings  on  Average  Supervisor  Rating,  CMD  RATE, 
and  the  NCO's  EER.  In  the  officer  sample,  Average  Supervisor  Rating  and  CMD 
RATE  had  fairly  high  loadings  on  one  factor  on  which  the  OER  rating  also  had  a 
moderate  loading  (.35).  These  results  indicate  that,  in  general,  the  average 
rating  measure  loaded  on  the  same  factor  as  did  other  supervisory  rating 
measures.  Unfortunately,  as  discussed  earlier,  these  other  rating  measures 
were  not  applicable  to  some  soldiers  and/or  had  high  proportions  of  soldiers 
with  missing  data  in  our  sample. 

Reliabilities  of  Alternate  Readiness  Composites 

The  Alpha  reliabilities  for  a  composite  composed  of  all  the  readiness 
measures,  including  the  Average  Supervisor  Rating,  were  obtained  for  E2-4 
personnel,  NCO's,  and  officers  separately  and  combined.  The  component  scores 
were  converted  to  standard  or  z-scores  prior  to  obtaining  the  reliabilities. 
The  IRR-Soldier  sampling  weights  were  applied  in  the  computations.  Table  15 
presents  the  reliabilities  for  when  the  component  scores  were  unweighted  in 
deriving  a  total  readiness  score  and  when  they  were  weighted  by  the  readiness 
workshop  weights.  The  reliabilities  were  quite  low.  Applying  the  readiness 
workshop  weights  to  the  component  measures  lowered  the  reliabilities  even 
further.  These  results,  when  considered  in  conjunction  with  the  factor 
analysis  results  and  the  missing  data  problems,  led  to  the  decision  to 
recommend  the  use  of  the  Average  Supervisor  Rating,  by  itself,  as  the  primary 
AFRP  measure  of  individual  readiness. 


Table  15 

Alpha  Reliability  of  Readiness  Composites  Using  All  Available  Component 
Measures 


E2-4 

NCO 

Officers 

All  Cases 

Unweighted  Composite 

.623 

.699 

.665 

.672 

Weighted  Composite 

.414 

.471 

.512 

.454 

This  is  not  to  say  that  AFRP  researchers  should  not  determine  the 
relationship  of  family  factors  and  related  phenomena  to  other  available 
performance/readiness  measures.  Alpha  reliabilities  were  obtained  for  four 
composites  that  were  suggested  by  the  results  of  the  factor  analyses.  These 
reliabilities  were  obtained  for  E2-4  enlisted  personnel,  NCOs,  and  officers, 
separately  and  combined.  The  component  measures  scores  were  converted  to 
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standard  or  z-scores  prior  to  obtaining  the  reliabilities.  As  can  be  seen  in 
Table  16,  the  reliabilities  were  generally  quite  low  for  three  of  these 
composites.  However,  the  reliability  of  the  fourth  composite,  Self-evaluation 
of  preparedness  for  a  nonconventional  war,  was  quite  respectable  (about  .90). 
This  composite,  which  may  reflect  soldier  confidence  to  handle  nonconventional 
warfare  situations,  may  be  shown  in  future  research  to  be  related  importantly 
to  fami ly  factors. 


Table  16 


Alpha  Reliabilities  of  Selected  Readiness/Performance  Composites 


Composite 

Component 

Measures 

E2-4 

NC0 

Officers 

All 

Cases 

Self-evaluation  of  prepared¬ 
ness  for  nonconventional  war 

S42  A,  B,  C 

.889 

.906 

.906 

.897 

Self-evaluation  of  prepared¬ 
ness  for  conventional  war 

S41 ,  S42D 

.573 

.611 

.669 

.604 

Disciplinary  problems 

S51 ,  S52 

.499 

.336 

.225 

.487 

No.  of  letters  and 
certificates 

S54,  S55 

.704 

.775 

.602 

.716 

Distribution  of  Average  Supervisor  Ratings 

The  percentage  distribution  of  the  Average  Supervisor  Ratings  in  the 
solider  population  was  estimated  using  the  Soldier-IRR  weights.  As  seen  in 
Table  17,  the  distribution  is  negatively  skewed  with  over  70%  of  the  weighted 
cases  having  scores  5.0  and  above.  The  mean  score  is  approximately  5.5  and 
the  standard  deviation  approximately  1.0.  This  distribution,  while  not  ideal, 
is  certainly  within  the  bounds  of  acceptability,  especially  when  considering 
the  high  concentrations  of  ratings  usually  found  at  the  upper  end  of 
operational  performance  scales. 
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Table  17 


Estimated  Percentage  Distribution  of  Average  Supervisor  Readiness  Ratings 


Average  Readiness 


Percentage 


Below 

1.99 

.1 

2.00  - 

2.49 

.7 

2.50  - 

2.99 

1.4 

3.00  - 

3.49 

2.1 

3.50  - 

3.99 

4.4 

4.00  - 

4.49 

7.3 

4.50  - 

4.99 

11.5 

5.00  - 

5.49 

16.3 

5.50  - 

5.99 

21.8 

6.00  - 

6.49 

20.5 

6.50  - 

7.00 

13.9 

100.0 
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Conclusion 


The  results  of  the  analyses  conducted  to  date  on  the  individual  readiness 
measures  point  to  the  pitfalls  of  combining  all  the  measures  into  one 
comprehensive  overall  index  of  readiness.  The  measures  are  tapping  different 
aspects  of  performance  as  evidenced  by  the  five  or  six  factors  that  emerged 
from  the  factor  analyses  of  the  E2-4,  NCO,  and  officer  data.  In  itself,  this 
result  is  not  too  worrisome,  but  substantial  amounts  of  data  are  missing  for 
some  key  measures,  e.g.,  the  unit  commander  ratings.  To  obtain  overall 
readiness  scores  that  would  be  comparable  across  all  the  sampled  soldiers,  a 
fairly  extensive  imputation  procedure  would  have  to  be  employed.  Moreover, 
such  a  composite  score  would  have  low  reliability  as  measured  by  the  Alpha 
coefficient. 

Instead,  the  results  lead  to  the  conclusion  that  the  supervisor  rating 
scales  should  be  used  as  the  primary  AFRP  measure  of  individual  readiness. 
Taken  as  a  set,  these  scales  were  assigned  the  highest  weight  by  the  readiness 
workshop  participants.  The  scales  are  apparently  measuring  one  underlying 
factor.  That  is,  there  is  every  reason  to  believe  that  readiness  scores  based 
on  the  scales  will  be  comparable  across  different  types  of  ratees.  The 
reliability  of  the  average  ratings  are  .90  or  higher,  and  the  distribution  of 
the  rating  scores  is  not  overly  concentrated  at  the  upper  end  of  the  scales. 

We  believe,  moreover,  that  the  missing  data  problem  for  the  ratings,  though 
serious,  has  been  adequately  overcome  through  the  part/whole  imputation 
procedure  used. 

The  use  of  the  rating  scales  in  further  readiness  research  should  be 
encouraged.  The  scales  could  even  be  used  operationally.  To  ease  the 
administrative  burden,  the  scales  could  be  reduced  in  number  by  one  scale  (to 
10  for  NCOs  and  officers,  6  for  E2-4)  without  unduly  lower'ng  the  reliability 
of  the  average  or  total  score.  To  further  ease  the  administrative  burden,  the 
scales  could  be  given  to  either  the  first-line  or  second-line  supervisors  to 
complete,  whichever  group  of  raters  is  more  accessible.  (We  would  not 
recommend,  however,  using  first-line  or  second-line  ratings  interchangeably.) 
The  arithmetic  average  of  the  ratings  would  yield  essentially  the  same  score 
as  a  weighted  average,  and  would  be  simpler  to  use.  But  perhaps  most 
important,  the  heavy  involvement  of  experienced  Army  NCOs  and  officers  in  the 
development  of  the  scales,  and  the  high  weight  given  the  scales  in  comparison 
to  many  other  measures  of  individual  readiness,  almost  assure  that  the  scales 
would  be  acceptable  to  the  Army. 
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APPENDIX  A 


Performance/Readiness 
Soldier  Questionnaire  Items 

and 

Individual  Readiness  Rating  Scales 


A- 1 


1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 


38. 


In  the  last  .month,  how  much  time  did  you  taka  off  from  duty  for  the  following  reasons?  (Please 
arrived  late,  or  left  early,  but  do  NOT  include  pass  or  leave  time.) 


count  time  when  you  were  sick. 


Does  Not 
Apply 

Did  Not 

If  Less  Than  One  Day 

How  Many  Hours? 

If  One  Day  or  More 
How  Many  Days? 

Taka  Off 

Any  Tima 

1 

2 

3 

4-6 

6-7 

1 

2 

3+ 

.  Problem  with  transportation  to 
duty  location  (for  example,  car 

wouldn't  start  or  bus  was  late) .... 

. o 

o 

o 

o 

o 

o 

o 

0 

o 

.  Nly  health  (for  example,  sick  call 
or  doctor/dentist  appointment)  . . . . 

. o 

o 

o 

o 

o 

o 

o 

o 

o 

Taking  care  of  child(ren) 
because  regular  care  was  not 
available . 

. © 

o 

o 

0 

o 

o 

o 

o 

o 

o 

Other  care  of  child(ren)  (for 

example,  sick  child  or  visit 
to  school) . 

. © 

o 

0 

0 

o 

o 

o 

o 

o 

o 

To  help  spouse  (for  example, 
take  spouse  to  doctor 
appointment) . 

. © 

o . 

o 

o 

o 

o 

o 

o 

o 

o 

To  take  care  of  personal  or 
family  business  (for  example, 
financial  matters  or  housing 
problems) . 

•-O 

o 

o 

o 

o 

o 

o 

0 

o 

Other  personal  or  family  reasons. . . . 

....o 

0 

o 

o 

o 

o 

o 

o 

o 

41.  If  we  were  to  go  to  war  today,  how  well  prepared  are  you  to 
perform  the  tasks  in  your  wartime  job?  (If  you  aren't  sure, 
give  your  best  estimate.) 

(_!  Very  well  prepared 
0  Well  prepared 

3  Neither  well  nor  poorly  prepared 
Poorly  prepared 
Very  poorly  prepared 


A- 3 


.  42.  How  well  pnpind  are  you  to  do  your  assigned  tasks  in  a 
conflict  in  which  the  enemy  uses  the  following  weapons? 
(If  you  aren't  sure,  give  your  best  estimate.)  (MARK  ONE 
CIRCLE  FOR  EACH  ITEM.) 


d.  Conventional  weapons .  O  O  O  O  O 


47.  What  is  the  senior  rater  overall  potential  box  check  on  your 
most  recent  NCO-ER? 


1  2  3  4  S 


Successful  Fair  Poor 


©  Does  not  apply:  I  have  not  been  rated  under  the  NCO-ER 
system  or  I  have  not  yet  received  a  copy  of  my  official  rating. 

*""■  )  48.  If  you  have  not  received  an  evaluation  under 

the  NCO-ER  system,  what  is  your  score  on 
your  most  recent  EER? 


43.  Compared  to  other  soldiers  in  your  same  pay  grade  in  your 
unit  or  place  of  duty,  how  would  you  rata  your  own  job 
performance? 


Much 

Better 

Than 

Most 

About 

Average 

Much 

Worse 

Than 

Most 

© 

0 

0 

® 

® 

® 

© 

44.  What  was  your  last  Physical  Readiness  Test 
Score? 


(Scoring  range  from  0-300.) 

O  Don't  know  score 


®®® 

©O© 

©@® 

®®@ 

0® 

0® 

0® 

0® 

0® 

..-.0® 


E 


SV.-T 

btoc i 


R':vr*.iyj;yir:w -.-I.?  "t*  ■ 


IF  ENLISTED,  CONTINUE. 

,  W  OFHCEFL,  SKIF  TO  QUESTION  4SL 


*&4 


46.  Have  you  ever  taken  a  Skill  Qualification  Test  (SQT)? 
©  Not  applicable 

O  Yes  Qno 


©  Does  not  apply;  I  have  not  been  rated  under 
the  EER  system. 


®  ®  ® 
©O© 
®@ 
®® 
®0 
®  © 
®® 
©0 
®® 


SKIP  TO  QUESTION  SI 

49.  What  is  the  senior  rater  potential  evaluation  box  check  of  your 
most  recent  OER7  (MARK  ONE  CODE) 


*46. 


What  was  your  most  recent  SQT 
score? 


O  Don't  know  score 


®  ®  ® 
©©© 
®@® 
®<*)® 
®0® 
®®® 
®  ®  ® 
00© 
®  ®® 
®  ®® 


@ 

50. 


51. 


IF  PAY  GRADES  E5-E9,  CONTINUE. 

IF  PAY  GRADES  E1-E4  SKIP  TO  QUESTION  51. 


Does  not  apply:  I  have  not  been  rated. 

Was  your  last  rating  in,  above,  or  below  the  center  of  mass  for 
your  senior  rater? 

O  Above  center  of  mass 
O  In  center  of  mass 
O  Below  center  of  mass 
O  Don't  know 

How  many  Articles  15  have  you  received  in  the  past  two 
years?  (MARK  ONE) 

Co 
O  1 
©2 
©3 

©  4  or  more 


I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  |  |  |  |  ■  I  I  I  I  I  I  I  I 


1 1 1 1 II 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1  ■  I  ■  1 1 1 1 1 

I 


52.  How  many  FLAG  Actions  (that  is,  auapanaion  of  a  favorabta 
personnel  action)  have  you  received  in  the  past  two  years? 
(MARK  ONE) 

Go 
O  i 
02 
03 

O  4  or  more 

53.  Have  you  ever  received  a  reduction  in  grade? 

O  Does  not  apply 

O  Yes  O  No 

The  next  two  questions  ask  about  the  number  of  Letters 
and/or  Certificates  of  Appreciation,  Commendation,  or 
Achievement  you  have  received  in  the  past  2  years.  DO 
NOT  count  letters  or  certificates  received  for  the  following. 

-  Completion  of  AIT  or  officer  basic  and  advanced 
training 

-  Completion  of  any  additional  training  courses 

-  Completion  of  Head  Start 

-  Announcement  of  a  promotion 

•  Announcement  of  an  award  or  decoration 

54.  How  many  Letters  of  Appreciation,  Commendation,  or 
Achievement  have  you  received  in  the  past  2  years? 

(MARK  ONE) 

O  0  0  6 

Ol  O  7 

0  2  O  8 

0  3  0  9 

0  4  O  io+ 

05 

55.  How  many  Certificates  of  Appreciation,  Commendation,  or 
Achievement  have  you  received  in  the  oast  2  years? 

(MARK  ONE) 

O  0  0  6 

Ol  O  7 

0  2  0  8 

0  3  0  9 

0  4  O  10+ 

05 

56.  How  many  awards  and  decorations  have  you 
received  during  all  your  time  in  the  military? 

(Include  all  badges  and  medals,  and  count  ones 
where  you  have  received  more  than  one  of  the 
same  type.) 


INDIVIDUAL  READINESS  RATING  SCALES 

Instructions 


Individual  rating  scales  will  be  used  to  assess  the  readiness  of  individual 
soldiers  participating  in  the  Army  Family  Research  Program.  Twelve  areas  of 
individual  readiness  have  been  lEbsled  and  defined  on  the  following  pages.  We 
would  like  you  to  use  these  scales  to  rate  one  or  more  soldiers  that  have  been 
identified  as  individuals  that  you  supervise. 


Because  many  soldiers  will  be  asked  to  rate  more  than  one  individual,  cards 
have  been  provided  to  facilitate  the  rating.  The  names  on  the  card  should  be  the 
same  as  the  names  in  the  box  on  page  4  of  the  individual  readiness  rating  booklet 
For  each  area  of  soldier  readiness,  place  your  card  on  the  form  so  that  the  first 
name  on  the  card  lines  up  with  the  first  row  of  numbers,  and  the  seco.ia  name  on 
the  card  lines  up  with  the  second  row  of  numbers,  etc.  as  in  the  example  below: 

EXAMPLE 

VIOMF  SB  JUURUSTl 


•((•ft 


UtittlnT 


Hakwa  llttla  wffort  Co  anaura 
job  gats  dona;  givma  up  aaslly 
whan  (scad  with  difficult  problams 
ruluetantly  aecapts  responsibility 
saldom  anticipates  problems. 


Puts  In  affort  and  Itaaps  trying 
whan  Its  wary  Important  to  conplata 
aaalgrusanta;  ovartonaa  most  obstacles; 
aceapta  responsibility  whan  glvan  It; 
antlclpatas  potential  problems. 


I 

Often  volunteers  to  work  extra  noursj 
pushes  hard  to  overcome  all  obstacle] 
readily  assumes  responsibility  when 
necessary;  identifies  and  attends 
to  potential  problems . 


Names  of  the  soldiers 
you  are  rating. 


I 


1 


3. 


1 


1 


5. 


1 


1 


1 . 


1 


3  4  3  4  7 
3  4  3  4  7 
3  4  5  4  7 
3  4  3  4  7 
3  4  5  4  7 
3  4  5  4  7 


3  4  3 

3  4  3 


The  process  for  completing  the  individual  readiness  scales  is: 

•  Each  area  of  individual  readiness  will  be  rated  on  a  7-point  scale. 

•  Each  scale  uses  statements  over  the  rating  scale  that  provide  examples  of 
the  kinds  of  behavior  covered  by  the  scale.  The  statements  also  describe 
different  levels  of  readiness. 

•  Ratings  should  be  based  on  how  reaay  the  individual  is  in  each  area  most 
of  the  time. 

•  Each  area  of  individual  readiness  is  a  relatively  independent  or  separate 
area.  Your  ratings  should  reflect  each  individual's  own  readiness  levels 
in  each  area  accurately. 


A- 6 


•  Each  individual  should  be  rated  independently  from  the  other  individuals 
in  each  area. 

•  Base  your  ratings  only  on  readiness,  not  on  unrelated  characteristics 
(for  example,  personal  appearance  or  rank) . 

Please  try  to  give  us  the  most  accurate  and  objective  ratings  you  can  give. 
If  you  have  any  questions,  please  ask  the  session  leader. 

Thank  you  for  your  cooperation. 
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APPENDIX  B 


Instructions  to  Readiness 
Weighting  Workshop  Participants1 


1  The  instruments  given  in  this  Appendix  were  given  to  officers  and  NCOs 
in  combat  military  occupational  specialties  (MOS).  A  parallel  set  of 
instructions  was  given  to  workshop  participants  in  combat  support  and  combat 
service  support  MOS. 
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GENERAL  INSTRUCTIONS 


During  this  workshop,  you  will  be  asked  to  make  a  series  of  judgments 
concerning  the  composition  of  overall  measures  of  individual  and  unit 
readiness.  All  your  judgments  will  concern  combat  soldiers  and  units.  As 
described  earlier,  the  Army  Family  Research  Project  is  collecting  six 
different  types  of  measures  that  reflect  the  readiness  of  individuals  and  two 
kinds  of  measures  that  reflect  the  capability  of  units  to  fulfill  their 
wartime  missions.  We  want  to  be  able  to  combine  these  various  measures  into 
one  comprehensi ve  overall  measure  of  individual  readiness  and  one  overall 
measure  of  unit  readiness.  To  do  this,  we  need  your  considered  judgments  as 
to  the  relative  importance  of  different  aspects  of  readiness  in  forming 
overall  scores  for  combat  soldiers  and  units. 

Since  we  have  many  different  kinds  of  measures,  we  have  tried  to  simplify 
your  judgmental  task  by  breaking  it  into  short  sets  of  judgments.  Twelve  nets 
of  judgments  will  involve  assigning  importance  or  relevance  weights  to 
measures  of  individual  readiness.  Five  sets  involve  assigning  weights  to 
measures  of  unit  readiness. 

To  further  simplify  your  judgmental  task,  we  will  use  the  same  weighting 
procedure  for  all  sets  of  measures.  This  procedure  has  been  used  successfully 
in  previous  Army  Research  Institute  research.  However,  in  order  to  apply  the 
procedure,  it  will  be  necessary  for  you,  the  judge,  to  first  carefully  examine 
the  items,  scales,  or  factors  that  are  candidates  for  inclusion  into  the 
readiness  composite. 

Please  do  not  be  concerned  that  some  of  the  individual  measures  reflect 
negatively  on  individual  or  unit  readiness,  e.g.,  duty  time  lost  for  personal 
reasons.  We  can  easily  adjust  these  types  of  measures  so  that  low  values 
indicate  higher  readiness.  Also  please  do  not  be  concerned  if  you  feel  that 
all  the  measures  of  a  particular  type  should  receive  low  or  zero  weights  in 
forming  a  readiness  composite.  You  will  have  an  opportunity  to  assign  all  the 
measures  of  that  type  a  low  weight  in  making  a  later  series  of  judgments.  As 
you  will  shortly  see,  the  judgment  procedure  allows  zero  weights  to  be 
assigned  any  measure  that  you  feel  should  not  be  part  of  an  overall  readiness 
measure.  The  procedure  also  readily  allows  you  to  assign  equal  or  tied 
weights  to  various  component  measures. 

Remember  use  your  best  judgment  at  all  times! 

Thank  you. 
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SPECIFIC  DIRECTIONS  FOR  ASSIGNING  READINESS  WEIGHTS 


The  procedure  for  assigning  weights  to  the  readiness  factors  is  as 
follows: 

1.  Please  carefully  examine  the  reasiness  measures  given  to  you.  Take  note 
of  the  content  of  the  items  or  scales  and  the  source  of  the  readiness 
information. 

2.  Rank  order  the  set  of  readiness  factors  tc  be  weighted  in  each  section  by 
assigning  a  "1"  to  the  most  important  for  an  overall  measure  of 
readiness,  a  "2"  to  the  next  most  important,  etc. 

3.  After  you  have  recorded  the  rank  orders  on  the  weighting  sheet,  assign 
100  points  to  the  factor  you  ranked  as  most  important  (the  factor  ranked 
#1).  Then  ask  yourself,  "If  I'm  assigning  100  points  to  this  readiness 
factor,  how  many  points  should  I  assign  to  the  next  most  important  one." 
If,  for  example,  you  think  that  the  second  most  important  one  should 
receive  half  the  weight  of  the  first,  assign  it  50  points.  Continue 
assigning  points  in  this  manner  until  all  the  factors  have  been  weighted. 

4.  In  assigning  the  points,  please  keep  in  mind  that  the  points  represent 
how  many  times  more  (or  less)  important  one  readiness  factor  is  than 
another.  For  example,  if  you  assign  30  points  to  one  factor  and  5  points 
to  another,  that  means  that  you  believe  that  the  30-point  factor  should 
receive  6  times  the  weight  in  a  total  readiness  score  as  the  5-point 
factor. 

5.  If  you  feel  that  two  or  more  factors  should  be  weighted  equally,  you  may 
assign  them  equal  weights.  For  example,  if  you  feel  that  the  factors 
ranked  first,  second,  and  third  are  really  tied  in  importance,  then  you 
can  assign  each  of  them  100  points. 

6.  If  you  believe  that  a  particular  readiness  factor  should  not  be  used  at 
all  in  arriving  at  the  overall  score,  you  should  assign  it  zero  points. 

7.  When  you  are  finished  assigning  points  to  all  readiness  factors,  make 
sure  that  they  are  in  correct  proportion  to  one  another.  That  is,  the 
whole  set  of  weights  accurately  reflects  the  relative  importance  of  each 
of  the  readiness  factors. 

8.  Please  use  the  space  at  the  bottom  of  each  weighting  form  to  record  any 
comments  you  may  have  about  the  weights  that  you  have  assigned.  We  would 
be  particularly  interested  in  your  reasons  for  assigning  some  readiness 
factors  or  measures  very  high  or  very  low  weights. 

9.  When  satisfied  with  your  assigned  weights,  please  proceed  to  the  next  set 
of  readiness  factors  to  be  weighted.  Once  you  begin  making  the  next  set 
of  judgments,  please  do  not  change  any  of  your  prior  judgments. 

10.  Remember  you  are  assigning  weights  to  form  overall  readiness  measures  for 
combat  soldiers  and  units. 

Thank  you  for  your  cooperation. 
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Self-Ratings  of  Readiness 
(Sets  la,  b  &  c) 


The  first  set  of  items  to  be  weighted  were  in  the  Soldier  Survey.  All 
sampled  soldiers  (officers  and  enlisted  personnel)  were  asked  to  rate  their 
own  degree  of  readiness.  The  specific  items  used  are  presented  below.  After 
examining  the  items,  please  assign  them  relative  weights  using  the  weighting 
procedure  described  earlier.  There  are  three  attached  weighting  sheets,  one 
for  assigning  weights  to  officers'  responses,  one  for  NCOs,  and  one  for  junior 
enlisted  personnel  (E2  to  E4).  Please  complete  all  three  weighting  sheets. 


41.  If  we  were  to  go  ta  war  today.  Sow  wed  prepared  are  you  to 
par  term  tha  tasks  in  your  wartime  job?  (If  you  aren’t  sure, 
gnra  your  Mat  estimate.) 

O  Vary  wai -prepared 
O  \AA*  prepared 

O  Maittier  well  nor  poorly  prepared 
o  Poorly  preoared 
O  Vary  poorly  prepared 


42.  How  well  prepared  are  you  ta  do  your  assigned  tasks  in  a 
conflict  in  which  the  enemy  uses  the  following  weapons? 
(If  you  aren't  sure,  give  your  best  sstimste.)  (MARK  ONE 
CIRCU  FOR  EACH  ITEM.) 


43.  Compared  to  other  soldiers  in  your  same  pay  grade  in  your 
unit  or  place  of  duty,  how  would  you  rate  your  own  job 
performance? 


Much 

Banar 

Than 

Most 


About 

Average 


Much 

Worse 

Than 

Most 


0 


0 


0 


•  ) 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Self-Ratings  of  Readiness 
(Officers) 

Rank 

Readiness  Factor  Order  Weight 

41.  Preparedness  for  wartime  job  _  _ 

42.  Preparedness  for  conflict  using 

a.  Nuclear  weapons  _  _ 

b.  Biological  agents  _  _ 

c.  Chemical  agents  _  _ 

d.  Conventional  weapons  _  _ 

43.  Comparison  of  job  performance 
with  other  soldiers 


Comments: 
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Name 


_  Workshop  _ 

Weighting  Sheet  for  Self-Ratings  of  Readiness 

(NCOs) 


Rank 

Readiness  Factor  Order  Weight 

41.  Preparedness  for  wartime  job  _  _ 

42.  Preparedness  for  conflict  using 

a.  Nuclear  weapons  _  _ _ 

b.  Biological  agents  _  _ 

c.  Chemical  agents  _  _ 

d.  Conventional  weapons  _  _ 

43.  Comparison  of  job  performance 
with  other  soldiers 


Comments: 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Self-Ratings  of  Readiness 
(E2  to  E4) 

Rank 

Readiness  Factor  Order  Weioht 

41.  Preparedness  for  wartime  job  _  _ 

42.  Preparedness  for  conflict  using 

a.  Nuclear  weapons  _  _ 

b.  Biological  agents  _  _ 

c.  Chemical  agents  _  _ 

d.  Conventional  weapons  _  _ 

43.  Comparison  of  job  performance 
with  other  soldiers 


Comments: 
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Self-Reports  on  Earlier  Performance  Measures 
(Sets  2a,  b  &  c) 


The  second  set  of  items  to  be  weighted  were  also  in  the  Soldier  Survey. 
The  sampled  soldiers  were  asked  to  indicate  their  earlier  performance  levels 
on  several  Army  measures.  The  specific  items  used  are  attached.  Note  that 
some  of  the  items  are  exclusively  for  officers  or  NCOs.  After  examining  the 
items,  please  assign  them  relative  weights  on  the  three  attached  weighting 
sheets  (for  officers,  NCOs  and  junior  enlisted,  respectively). 
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47.  What  la  the  senior  rater 
most  meant  NCO*ER? 


overall  potential  boa  check  on  your 


<V  Cn  O 


Does  not  apply.  I  have  not  burn  rated  under  the  NCO-ER 
system  or  I  have  not  yet  received  a  copy  of  my  oftiui  rating 


44.  What  waa  your  laet  Physical  Readiness  Teat 
Score? 


(Scaring  range  from  0*300.) 


O  Don  *  know  score 


>  48.  M  you  have  not  received  an  evaluation  under 
the  NC0*ER  system,  what  is  your  score  on 
your  most  recent  EER? 


A  Does  not  apply:  I  have  not  been  rated  under 
the  EER  system 


SKIP  TO  QUESTION  51 


49.  What  la  the  senior  rater  potential  evaluation  box  check  of  you 
most  recent  0ER7  (MARK  ONE  CODE) 


IF  ENUSTED.  CONTINUE. 

_ IF  OFFICER.  SKIP  TO  QUESTION  49. 

46.  Have  you  ever  taken  a  Skill  Qualification  Teat  (SQT)? 
fu*  Not  applicable 


C  Yes 

u 


48.  Whet  waa  your  most  recent  SQT 
score? 


Don  t  know  score 


i 

m  ! \ 

h*  Does  not  apply:  I  have  not  been  rated 

iO.  Wes  your  last  rating  in.  above,  or  below  the  center  of  mass  fc 
your  senior  rater? 

Above  center  of  mass 
*w.'  In  center  of  mass 
Below  center  of  mass 
Don  t  know 

>1.  How  many  Articles  15  have  you  received  in  the  past  two 
f  years?  (MARK  ONE) 


IF  PAY  GRADES  E5-E9.  CONTINUE. 

IF  PAY  GRADES  E1-E4  SKIP  TO  QUESTION  51- 


'  A  or  more 
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52.  How  many  FLAG  Action*  (that  >*.  suspension  o i  a  favorable 
pataonnai  action)  hava  you  racaiwad  in  tha  pact  two  years? 
(MARK  ONE) 

Oo 

Oi 

02 

03 

O  4  or  mors 

53.  Hava  you  evar  racaivad  a  raduction  in  grada? 

O  Ooa*  not  apply 

O  Yea  O  No 

Tha  noxt  two  questions  ash  about  tha  number  of  Letters 
and/or  Certificates  of  Appreciation.  Commendation,  or 
Achievement  you  have  received  in  the  oast  2  veers.  OO 
NOT  count  letters  or  certificates  received  for  the  following. 

•  Completion  of  AIT  or  officer  basic  and  advanced 
training 

•  Completion  of  any  additional  training  courses 

•  Completion  of  Head  Start 

-  Announcement  of  a  promotion 

•  Announcement  of  an  award  or  decoration 


54.  How  many  Letter*  of  Appreciation.  Commendation,  or 
Achievement  have  you  received  in  the  part  2  years? 

(MARK  ONE) 

o  O  a 

1  O  7 

2  ~j  8 

3  ;  9 

4  •  'j  10+ 

5 

c*  marry  Certificate:  A-~-ireciation.  Commendation,  or 

Achievement  have  you  received  in  the  oast  2  veers? 

(MARK  ONE) 

r:  o  ' ;  s 

O  i  ;■  7 

O  2  O  a 

~J  3  9 

4  10+ 

< 5 

56.  How  many  awards  and  decorations  have  you 
received  during  all  your  time  m  the  military  ? 

(Include  all  badges  and  medals,  and  count  ones 
where  you  have  received  more  than  one  of  the 
seme  type.) 


(iya 

r, 

0 

(b 

■i) 

(  3} 

V 

(4. 

(t. 

i) 

(ti 

IV 

■7) 

‘•'’■j 

til 

o 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Self-Reports  on  Earlier  Performance  Measures 

(Off icers) 


Readiness  Factor 


Rank 

Order  Weight 


44.  Last  Physical  Readiness  Test  Score 

49.  Most  recent  OER  evaluation 

50.  Position  in  senior  rater  mass 

51.  Articles  15  in  past  two  years* 

52.  FLAG  Actions  in  past  two  years* 

54.  Letters  of  Appreciation,  etc. 

in  past  two  years 

55.  Certificates  of  Appreciation,  etc. 

in  past  two  years 

56.  Awards  and  decorations  received 

in  the  military 


*  Will  be  scored  so  that  low  values  indicate  greater  readiness. 
Comments: 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Self-Reports  on  Earlier  Performance  Measures 

(NCOs) 


Read i ness  Factor 


Rank 

Order  Weight 


44.  Last  Physical  Readiness  Test  Score 

49.  Most  recent  OER  evaluation 

50.  Position  in  senior  rater  mass 

51.  Articles  15  in  past  two  years* 

52.  FLAG  Actions  in  past  two  years* 

54.  Letters  of  Appreciation,  etc. 

in  past  two  years 

55.  Certificates  of  Appreciation,  etc. 

in  past  two  years 

56.  Awards  and  decorations  received 

in  the  military 


*  Will  be  scored  so  that  low  values  indicate  greater  readiness. 
Comments: 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Self-Reports  on  Earlier  Performance  Measures 

(E2  to  E4) 


Readiness  Factor 


Rank 

Order  Weight 


44.  Last  Physical  Readiness  Test  Score 

49.  Most  recent  OER  evaluation 

50.  Position  in  senior  rater  mass 

51.  Articles  15  in  past  two  years* 

52.  FLAG  Actions  in  past  two  years* 

54.  Letters  of  Appreciation,  etc. 

in  past  two  years 

55.  Certificates  of  Appreciation,  etc. 

in  past  two  years 

56.  Awards  and  decorations  received 

in  the  military 


*  Will  be  scored  so  that  low  values  indicate  greater  readiness. 
Comments: 
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Supervisory  Ratings  of  Readiness 
(Sets  3a,  b,  &  c) 


Each  of  the  sampled  soldiers  was  rated  by  his/her  first  and  second  line 
supervisors.  The  officers  and  NCOs  were  generally  rated  on  12  readiness 
scales,  4  of  which  involve  supervisory  skills.  The  junior  enlisted  personnel 
(E2  to  E4)  were  generally  rated  on  only  8  of  the  12  scales.  Examples  of  the 
scales  used  are  attached.* 

The  first  and  second  line  supervisors  made  their  evaluations  of  a  given 
soldier  independently.  The  readiness  score  for  a  sampled  soldier  on  any  one 
of  the  rating  scales  will  be  the  average  of  the  first  and  second  line 
supervisors'  ratings. 


*  See  Appendix  A  for  the  set  of  scales. 
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Name 


_  Workshop  _ 

Weighting  Sheet  for  Supervisory  Ratings  of  Readiness 

(Officers) 


Rank 

Readiness  Factor  Order  Weight 

Cooperation/Teamwork/Esprit'  De  Corps  _  _ 

Effort  and  Initiative  _  _ 

General  Soldering  Skills  _  _ 

Individual  Deployability  (Army  Task/Mission)  _  _ 

Individual  Deployability  (Personal/Family)  _  _ 

Job  Discipline  _  _ 

Job  Technical  Knowledge/Skills  _  ___ 

Performance  Under  Pressure  and  Adverse  Conditions  _  _ 

Care  and  Concern  for  Subordinates  _  _ 

Care  and  Concern  for  Subordinates'  Families  _  _ 

Leadership  of  Subordinates  _  ___ 

Maintaining  Training  Status  of  Subordinates  _  _ 


Comments: 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Supervisory  Ratings  of  Readiness 

(NCOs) 

Rank 

Readiness  Factor  Order  Weight 

Cooperation/Teamwork/Esprit'  De  Corps  _  _ 

Effort  and  Initiative  _  _ 

General  Soldering  Skills  _  _ 

Individual  Deployability  (Army  Task/Mission)  _  _ 

Individual  Deployability  (Personal/Family)  _  _ 

Job  Discipline  _  _ 

Job  Technical  Knowledge/Skills  _  _ 

Performance  Under  Pressure  anJ  Adverse  Conditions  _  _ 

Care  and  Concern  for  Subordinates  _  _ 

Care  and  Concern  for  Subordinates'  Families  _  _ 

Leadership  of  Subordinates  _  _ 

Maintaining  Training  Status  of  Subordinates  _  _ 


Comments : 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Supervisory  Ratings  of  Readiness 

(E2  to  E4) 

Readiness  Factor  Order 

Cooperation/Teamwork/Esprit'  De  Corps  _ 

Effort  and  Initiative  _ 

General  Soldering  Skills  _ 

Individual  Deployability  (Army  Task/Mission)  _ 

Individual  Deployability  (Personal/Family)  _ 

Job  Discipline  _ 

Job  Technical  Knowledge/Skills  _ 

Performance  Under  Pressure  and  Adverse  Conditions 


Rank 

Weight 


Comments: 
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Overall  Readiness  Measures 


(Sets  4a,  b,  &  c) 


In  addition  to  the  supervisory  and  self-ratings  and  the  self-reports  on 
earlier  performance,  tf ree  other  types  of  readiness  measures  may  be  used  in 
arriving  at  an  overall  comprehensive  measure  of  readiness: 

1)  A  measure  of  promotion  rate  based  on  data  in  the  Officer  or  Enlisted 
Master  File; 

2)  An  overall  assessment  of  each  sampled  soldier's  job  performance 
given  by  the  soldier's  unit  commander;  and 

3)  The  amount  of  time  in  the  last  month  that  the  soldier  reported 
taking  off  from  duty  (not  including  pass  or  leave  time). 

Further  descriptions  of  these  three  additional  types  of  readiness  measures  are 
attached.  Please  examine  them  carefully  before  assigning  the  last  sets  of 
weights  for  determining  overall  individual  readiness  for  officers,  NCOs,  and 
E2  to  E4  soldiers.  Also  please  review  the  content  of  the  three  previous  sets 
of  readiness  measures. 
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PROMOTION  RATE 


Promotion  rate  will  be  obtained  through  first  calculating  the  amount  of 
time  each  soldier  spent  at  each  previous  grade  level.  Next  these  times  will 
be  averaged  to  obtain  a  mean  time  within  grade  for  each  soldier.  Finally,  the 
mean  time  within  grade  for  all  sampled  soldiers  at  the  same  current  grade  will 
be  compared  and  determination  made  as  to  whether  each  soldier  is  being 
promoted  at  a  faster  or  slower  rate  than  other  soldiers  of  the  same  grade. 

This  means,  for  example,  that  majors  will  be  compared  with  other  majors, 
captains  with  captains,  etc.  Special  adjustments  will  be  made  for  soldiers 
who  received  a  reduction  in  grade. 


B-22 


UNIT  COMMANDER'S  RATINGS 


The  commander  of  each  unit  was  given  a  list  of  all  the  soldiers  sampled 
from  within  the  unit.  The  list  contained  on  the  average  about  30  soldiers. 

The  commander  was  asked  to  assess  each  soldier's  job  performance  using  the 
rating  scale  shown  below.  Since  some  of  the  officers  assigned  to  some  units 
had  higher  grades  than  the  unit  commander,  the  commander  could  decline  to  rate 
these  officers. 


SOLDIER  JOB  PERFORMANCE 

Overall,  the  command's  assessment  of  the  soldier's  job  performance  is 
that  this  soldier  is  ... 


One  of 
the 

Best 

Above 

Average 

Average 

Below 

Average 

One  of 
the 
Worst 

01 

02 

03 

04 

05 
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TIME  OFF  DUTY  LAST  MONTH 


The  amount  of  time  taken  off  from 
from  Question  38  of  the  Soldier  Survey 
readiness  measure,  low  amounts  of  time 
greater  readiness. 


duty  in  the  last  month  will  be  derived 
(see  below).  In  arriving  at  an  overall 
taken  off  from  duty  will  indicate 


38.  In  the  last  month,  ho**  much  timo  did  you  tako  art  from  duty  for  tho  Mooing  racoons?  (Ptooso  count  um  when  you  wore  sick, 
armrad  lata,  or  loft  sarfy.  but  do  NOT  inckida  paas  or  taava  time.) 


a.  Problem  with  transportation  to 
duty  locaoon  (for  example,  car 
wouldn't  start  or  bus  was  lata) 

b.  My  health  (for  example,  sack  call 


or  doctor/omts  ippmnn>«m . vj 

c.  Taking  care  of  chaidiran) 
because  regular  care  was  not 

available . ©  O 

d.  Other  care  of  cfnld<ren)  (for 
example,  sack  child  or  vort 

to  school) . ©  O 

e.  To  help  spouse  (for  example, 
take  spouse  to  doctor 

appointment) . ©  O 


f  To  take  care  of  persona,  r 
family  business  (for  exam  ole. 
financial  matters  or  housing 

problems) . O 

g.  Other  personal  or  family  reasons . O 


OWNot 
Taka  Off 
Any  Time 


If  Leas  Than  One  Day 
How  Many  Hours? 


1  2  3  M  *-7 


o  o  o  o  o 

o  o  o  o  o 

o  o  o  o  o 

o  o  o  o  o 

o  o  o  o  o 

o  o  o  o  o 

o  o  o  o  o 


If  One  Day  or  More 
Ho**  Many  Days? 


1 _ 2  3* 


GOO 

o  o  o 

o  o  c 

o  o  o 

c  o  o 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Overall  Readiness  Measures 

(Officers) 


Rank 

Readiness  Factor  Order  Weight 

Self-Rating  Composite  _  _ 

Self-Report  Composite  _  _ 

Supervisory  Rating  Composite  _  _ 

Promotion  Rate  _  _ 

Unit  Commander  Rating  _  _ 

Time  Off  Duty  Last  Month*  _  _ 


*  Will  be  scored  so  that  low  values  indicate  greater  readiness. 
Comments: 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Overall  Readiness  Measures 

(NCOs) 

Rank 

Readiness  Factor  Order  Weioht 

Self-Rating  Composite  _  _ 

Self-Report  Composite  _  _ 

Supervisory  Rating  Composite  _  _ 

Promotion  Rate  _  _ 

Unit  Commander  Rating  _  _ 

Time  Off  Duty  Last  Month*  _  _ 


*  Will  be  scored  so  that  low  values  indicate  greater  readiness. 
Comments: 
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Name  _  Workshop  _ 

Weighting  Sheet  for  Overall  Readiness  Measures 

(E2  to  E4) 


Rank 

Readiness  Factor  Order  Weight 

Self-Rating  Composite  _  _ 

Self-Report  Composite  _  _ 

Supervisory  Rating  Composite  _  _ 

Promotion  Rate  _  _ 

Unit  Commander  Rating  _  _ 

Time  Off  Duty  Last  Month*  _  _ 


*  Will  be  scored  so  that  low  values  indicate  greater  readiness. 
Comments: 
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